RLVR's Diversity Collapse Concentrates Before the First Arithmetic Step, 11x to 16x Shift

Found first: a primary source the press has not covered yet.

RLVR's solution-space collapse happens before the chain of thought begins. Per-token likelihood shifts at the reasoning entrance run 11x to 16x larger than during all downstream steps. The paper, posted to arXiv on 29 August 2026, reports that solution coverage falls by up to 67% after RLVR training and identifies that entrance as both the primary site of loss and a viable target for recovery.

What the source says

Authors Qiancheng Zhou and Ruizhe Li used the Countdown task, which has an exhaustively enumerable solution space, then extended the analysis to six math benchmarks with 7B and 14B models. They tested PPO on Qwen2.5-3B and GRPO on Qwen2.5-3B-Instruct. Supplying the entrance prefix alone restores completion rates from 0.018 to 0.212 under PPO, an increase of over an order of magnitude. Late-layer parameter interpolation with early checkpoints recovers 37% of lost coverage at no loss in pass@1. An SFT baseline preserves more than double the coverage that RLVR-trained models retain.

Why it matters

The result pins the tradeoff to a specific location. RLVR's accuracy gains carry a coverage penalty, but that penalty is concentrated at the reasoning entrance while the chain itself remains largely intact. Entrance-targeted interventions are therefore a credible partial fix: the prefix restoration and parameter interpolation in this paper both act at the identified site. For anyone relying on test-time sampling to scale inference compute, coverage loss is now partially recoverable without sacrificing the single-sample accuracy RLVR provides. The SFT comparison gives a concrete cost: standard fine-tuning holds more than twice the solution coverage.