Ataraxos beats Stratego champion on $8,000 training budget

Ataraxos, a new AI system described in Nature, beat Pim Niemeijer, who the paper calls the most decorated Stratego player in history, 15 games to 1 with 4 draws. The training run that produced it cost roughly $8,000, according to the paper.

DeepMind's DeepNash, the prior superhuman Stratego agent, cost between $3 million and $4.5 million by the team's reckoning. Ataraxos used about 1/500th of that compute and 1/30th of the training games, and the authors call the result "the first superhuman result in the game's history."

Stratego hides a lot. More than 10³³ opening configurations are possible, and most opponent pieces stay unknown for most of the game. The system wraps three parts together: a policy-value network trained by self-play, a belief network modelling what the opponent could be hiding, and a test-time search procedure. The authors tune "a coordination between regularization strength and policy update size" through training to keep learning stable.

The same recipe runs on games that look nothing like Stratego. In Hanabi, the fully cooperative card game, Ataraxos averages 24.654 points in the two-player version with 77.53% perfect games, and 24.410 in five-player with 58.05% perfect. In Dou Dizhu, a team-based Chinese card game, it beats PerfectDou and DouZero with statistical significance. In Barrage Stratego, a smaller variant, it defeated the three top-ranked human players across multiple 50-game series.

"The success of this approach across adversarial, cooperative and team games establishes a design pattern for reinforcement learning and search that is effective under large amounts of hidden information," the authors write.

The compute bill is still a research-lab bill. Training used 16 NVIDIA H100 GPUs for a week plus 4 more H100s for four days on the belief network, across 163 million finished games and 208 billion environment steps. The paper does not report a head-to-head match against DeepNash.