DataFlex-RL study: no data policy beats uniform GRPO sampling

Uniform sampling won. In a paired-seed evaluation of thirteen data policies for reinforcement learning with verifiable rewards on Qwen2.5-7B-Base, across twelve mathematics, logic, and science benchmarks, none of eight rollout-selection or reweighting methods and none of three adaptive mixtures beat plain uniform GRPO at 95% confidence. That is the flat finding of DataFlex-RL, a new evaluation-platform paper on arXiv.

The training itself worked. "Uniform GRPO improves the domain-balanced average accuracy by 7.76 percentage points over the untrained checkpoint," the paper reports. It is only the fancier data policies layered on top that failed to move the paired confidence interval.

A corrected twelve-seed extension on Llama-3.1-8B-Base did not surface a consistent winner either. And when the authors rescored nine Qwen2.5-7B-Instruct runs on a math-heavy six-benchmark summary (five mathematics benchmarks plus GPQA-Diamond, no logic benchmark) against the balanced twelve-benchmark summary, the two rankings came out negatively correlated, with a correlation coefficient of -0.33. Change which benchmarks you weight, and the leaderboard can flip.

"Across the controlled settings studied here, changing the data policy measurably changes the training process but does not produce a reproducible improvement over uniform training," the authors write.