RIPO Claims 60% Gain Over GRPO on AIME24 in ICML Paper

A new arxiv paper accepted at ICML 2026 argues the reinforcement learning algorithm behind most modern reasoning-model training has a specific geometric flaw, and that patching it can produce up to a 60 percent improvement over GRPO on the AIME24 math benchmark.

The flaw the authors describe is concrete. PPO-Clip, the workhorse behind RL post-training, implicitly measures how far a new policy has moved from the old one using a Euclidean metric. Their claim is that policy spaces are not Euclidean, they are Riemannian manifolds, and this mismatch produces "overly conservative updates in low-probability regions while aggressive in high-probability regions." The consequence they name is exploration collapse, meaning the model stops trying novel reasoning paths and stalls on hard problems. Their proposed fix, Riemannian Isometric Policy Optimization, or RIPO, replaces the Euclidean clipping with an isometric update on the Riemannian manifold, and the paper reports gains across seven competition-level benchmarks.

Why this matters if you are not training models yourself. The RL step that turns a base model into a reasoning model is one of the most expensive and least well-understood parts of the frontier training stack. If a purely algorithmic swap really delivers this kind of lift on a hard math benchmark, every open source and closed lab doing PPO-style post-training has a cheap experiment to run before their next reasoning-model release, not after.

The honest caveat is that this is a single paper's claim, the "up to 60 percent" number is on one benchmark, and the abstract does not disclose the base model, compute overhead, or how the gains distribute across the other six benchmarks it references. Take the direction as suggestive, not settled, until an independent lab replicates on their own training run. The forward-looking part is that unlike a data or scale advantage, an algorithmic geometry fix is the kind of change an open source RL framework can add in weeks, so if it holds up you will see it show up in the standard tooling fast, and the reasoning-model gap between well-funded and merely-funded labs narrows a bit.