Agon paper: rival-model RL doubles GRPO pass@1 on DeepMath

Every dominant reasoning model right now is trained with some flavour of verifiable-reward RL, GRPO being the standard, and the honest problem with that recipe is well known inside labs: the reward looks at the final answer, not the reasoning trace, so on hard problems the model learns to write more rather than to think better. A new arxiv preprint from Vladislav Beliaev, posted on 8 July 2026, proposes a structural fix called Agon.

The mechanism is competitive rather than supervised. Two comparably strong models attempt the same problem, and in alternating roles one drafts a solution while the other reads that draft and then solves the problem itself. Each is rewarded for out-solving the other. To win, the paper argues, a model has to out-reason a rival that has already seen its work, so reasoning gets judged implicitly during training. There are no process labels and no separate reward model, which is the part that matters if you have been paying for either.

The headline claim is that on the hard split of DeepMath with Qwen3, the setup doubles GRPO's pass@1, and that this is roughly eight times the gain of an untrained Mixture-of-Agents pass over the same base. The abstract says the ordering replicates on competitive-programming code and across Qwen3.5 and Gemma 4. At inference the pair deploys as it trains, a two-stage cascade in which one model drafts and the other answers after reading the draft.

The honest caveats are the obvious ones for a single-author preprint. The abstract does not give absolute pass@1 numbers, so 'doubles' is a ratio without a floor, and the results cover math and code rather than open-ended reasoning. What the reporting doesn't give you is the compute cost of running two model updates in parallel, or how the method behaves as the two models converge in strength. If the result does hold up in independent replications, the payoff for open-model teams is direct: a route to reasoning gains that does not require process supervision or a separately trained reward model, which are two of the more expensive pieces of the current frontier recipe.