IER Selector Matches Full On-Policy Distillation at 1% Tokens

Supervising 0.1% to 1% of student-generated tokens can match or beat teacher supervision applied to every token, according to a preprint posted to arXiv by Huanxin Sheng, Zhiling Ye, Haonan Wang, Jian Wang, Jinjie Gu and Jian Kang.

The setup is sparse on-policy distillation. A student model generates a trajectory; a teacher supervises only a subset of those tokens. 'Useful teacher guidance can yield a noisy update when its gradient is estimated from a sampled next token,' the authors write. Which tokens are worth supervising is not just a question of usefulness. It is also a question of whether the gradient estimate at that token is reliable enough to trust.

The paper's answer is a metric called the information-efficiency ratio, or IER, built from a signal-to-noise decomposition of gradient estimation error at a fixed prefix under an optimal scalar baseline. A candidate-set approximation lets IER be used as a token selector, on its own or combined with existing usefulness scores, while keeping the sampled reverse-KL training objective.

On mathematical and medical reasoning tasks, the authors report that IER 'improves existing selectors in multiple settings,' with 'sparse configurations matching or exceeding full OPD without token selection at small token budgets of 0.1%-1%.'

No per-benchmark accuracy numbers appear in the abstract, and it does not name the teacher and student model pair used. The paper sits alongside a run of related distillation and self-improvement work we have been tracking this week, including onPanda's reported 52% annotation cut and RRSI's 30% token savings.