DiagEvo Beats R-Zero by 4.5 Points on Qwen3-8B Math

DiagEvo, a diagnosis-guided self-play framework, reaches 72.3% mean accuracy across five mathematical reasoning benchmarks on Qwen3-8B, 4.5 percentage points above R-Zero. Across all nine benchmarks the paper reports 57.4% mean accuracy, 1.1 points above DARC, using a default 4B diagnostician model.

The mechanism is a diagnostician that mines the solver's own failure history for recurring error causes and stores them in a hierarchical memory, grouping related causes under skill nodes and tracking each as "Active" or "Mastered" based on self-consistency on targeted questions. A challenger then balances cause-targeted question generation with free exploration, while "Double-confidence filtering retains intermediate-difficulty questions only when the most common solver answer has a clear vote lead."

The authors argue that unguided self-play plateaus because signals like difficulty, learnability, or diversity keep questions varied but "do not specify which unresolved reasoning weaknesses later rounds should target," and that the direction can be derived from the solver's own errors instead. Ablations credit both the hierarchical memory and the double-confidence filter for the gains. It lands in a busy stretch of fine-tuning research our tracker has been logging. Yesterday brought NoRA, a normalized LoRA method reporting faster convergence and less catastrophic forgetting.