Andrés Monroy-Hernández reposted
George Pólya famously said that `it is better to solve one problem five different ways than five problems one way.' He'd be sad to learn what we found: RLVR pushes LLMs in exactly the opposite direction, toward solving each problem in just one way! 📜: arxiv.org/abs/2610.11064
AI Weekly's analysis
→
- RLVR post-training concentrates probability onto fewer correct answer modes even as accuracy holds or improves, the paper reports.
- Frontier models already show highly concentrated solution distributions before any further RLVR fine-tuning, according to ModeBench measurements.
- The proposed Re:Max method keeps one verified example per mode in a replay buffer, improving both success and solution diversity.
Read full analysis →
View on Bluesky →
