Waypoint-OPD gates distillation using mid-run teacher checkpoint
TL;DR
- Waypoint-OPD gates endpoint policy-shift distillation using an intermediate teacher checkpoint, suppressing the shift when early and late RL updates disagree in direction.
- With RL-trained Qwen3-4B teachers and Qwen3-8B, Qwen3-14B, and Qwen3-30B-A3B students, the method tops domain averages in all six math and code settings.
- Gating adds only one extra frozen-teacher scoring pass over endpoint-only transfer, and ablations show that removing or shuffling gates reduces math reasoning.
Waypoint-OPD pulls a mid-training snapshot of a reinforcement-learned teacher and uses it to decide when the teacher's final policy shift is actually worth copying to a student. The paper, posted to arXiv on 15 Oct 2026, notes that 'the same endpoint shift can result from early and late shifts that reinforce or partially cancel each other,' and gates the endpoint supervision when the two shifts at an intermediate checkpoint point in opposing directions.
RL-trained Qwen3-4B teachers are tested against Qwen3-8B, Qwen3-14B, and Qwen3-30B-A3B students on math and code reasoning. 'In all six domain--student settings, Waypoint-OPD achieves the highest domain-average score among the compared distillation methods,' the authors report, and the gating costs only 'one additional frozen-teacher scoring pass' beyond endpoint-only transfer. Ablations on the 8B student show that removing or shuffling the gates reduces mathematical reasoning performance. Code is posted at GitHub.
The abstract does not publish raw benchmark scores, per-setting deltas, or the list of baselines compared. It lands amid a run of post-training work we have been following across open-source and fine-tuning coverage this week.
Originally reported by arxiv.org
Read the original article →Original headline: Alibaba Cloud's Waypoint-OPD Gates RL Distillation Using Mid-Run Teacher Checkpoints, Tops Six Qwen3 Settings