GEPA prompt optimizer beats GRPO by 6%, uses 35x fewer rollouts

A prompt optimizer called GEPA, short for Genetic-Pareto, matches or beats reinforcement-learning fine-tuning on six tasks while using up to 35 times fewer rollouts, according to a preprint on arXiv from a group whose authors include Omar Khattab, Matei Zaharia, Ion Stoica and Dan Klein. Two researchers we track in our Who's Who circulated the paper.

The method adapts LLM-based systems by sampling trajectories of reasoning, tool calls and tool outputs, then reflecting on them in natural language to critique and rewrite the prompts driving them. The framing is a direct challenge to methods like Group Relative Policy Optimization, which the authors describe as "often require thousands of rollouts to learn new tasks."

On the headline comparison, the paper reports GEPA "outperforms GRPO by 6% on average and by up to 20%, while using up to 35x fewer rollouts." Against the leading prompt-optimization baseline, it "outperforms the leading prompt optimizer, MIPROv2, by over 10% (e.g., +12% accuracy on AIME-2025)." The authors also report "promising results as an inference-time search strategy for code optimization."

The underlying claim is a bet on interpretability as a training signal. "The interpretable nature of language often provides a much richer learning medium for LLMs, compared to policy gradients derived from sparse, scalar rewards," the paper argues. The abstract does not disclose which base models were used or how reflection cost trades against the rollouts it saves.