Evolution Strategies lift LLM pass@k where RL narrows it

Standard reinforcement learning post-training, the paper argues, buys accuracy at the cost of coverage.

In a preprint on arXiv, Conor F. Hayes, Elliot Meyerson, Kajetan Schweighofer, Roberto Dailey, Babak Hodjat, Risto Miikkulainen and Xin Qiu take aim at the default post-training recipe for reasoning-focused LLMs. Under RL, they write, "the model's output distribution narrows around high-reward outputs, causing the solution coverage to collapse." That matters most in pass@k, the test-time regime where a model is asked for multiple candidate answers and judged on whether any of them is right — the setting the authors frame as central to discovery work in math and science.

Their alternative is Evolution Strategies, which they describe as "a population-based, gradient-free post-training method that optimizes directly in weight space through random perturbations." The headline claim is that "ES achieves consistently higher pass@k than RL and produces a broader output distribution with greater solution coverage," and that the extra coverage carries over to standard math benchmarks.

The abstract names no specific RL baseline, no model sizes, and no benchmark figures. What it does commit to is the title's bet: for problems where diverse candidate solutions are the whole point, "ES provides a better foundation for post-training" than RL.