O³PD: order-only distillation beats OPD by 5.6 on AIME25

Training a student language model to replicate only the order of the teacher's top-K tokens, ignoring probability values entirely, outperforms standard on-policy distillation by up to 2.9 average points, and 5.6 points on AIME25, across six teacher-student pairs on math-reasoning benchmarks. That is the central claim of Order-Only On-Policy Distillation, a paper from Guangzeng Han, Yu Wu and Xiaolei Huang proposing a method they call O³PD.

The argument is that teacher token probabilities entangle two signals: which tokens the teacher prefers, and how confident it is. "The latter reflects the teacher's own capacity and calibration, which a smaller student may be unable to reproduce," the authors write, which also makes OPD sensitive to the temperature choice because temperature rescales probabilities without changing their order. O³PD keeps only the ranking, so the objective is "invariant to the teacher temperature and needs no temperature tuning."

The lift is not specific to the exact loss they picked. "Other ranking losses yield similar gains, so the benefit comes from ordinal supervision itself," the paper reports. Analysis shows O³PD-trained students depart further from teacher probabilities than OPD "yet reproduces its ranking more faithfully and yields a stronger student," which the authors read as evidence that ranking, not magnitude, is the channel for knowledge transfer. The evaluation is narrow: three math-reasoning benchmarks and six pairs, with no reported numbers on coding, factual QA, or instruction following.