EDEN perturbs rollout embeddings, outperforms GRPO on math

The paper names a specific failure mode. In reinforcement learning with verifiable rewards, 'repeated sampling from the same policy can yield highly similar trajectories, reducing within-group reward variation and weakening group-relative advantage signals.' Fixes so far have reached for higher temperature, entropy bonuses, asymmetric clipping, or just more rollouts. EDEN tries something cheaper.

The method, laid out in a preprint from Daijiao Liu, Huiyi Wang, Lina Yao and Dong Gong, replaces a small fraction of each rollout group with rollouts the same policy generates from the prompt plus 'a single appended, perturbed embedding position, rather than from the prompt alone.' Gaussian noise drives the variation; a shared anchor keeps the perturbed group pointing in a common direction. 'Selected perturbed rollouts contribute to GRPO updates with downweighted advantages,' and the architecture, decoding and test-time inference, the authors write, 'remain unchanged.'

Across model scales and mathematical reasoning benchmarks, the paper reports that EDEN 'outperforms GRPO and the exploration controls evaluated in the same pipeline, with the largest gains in sampled solution coverage.' Analyses also claim that 'more prompt groups provide non-zero group-relative learning signals and that noise and anchoring play complementary roles.'

The abstract publishes no per-benchmark accuracy numbers, no model-size table, and no replacement-fraction value; those specifics sit in the body of the paper rather than its summary.