One Query Delivers 71.5% Coverage in On-Policy LLM Distillation

A single training query gets you 71.5% of the state-space coverage that a full dataset delivers for on-policy distillation of LLMs, according to a new arxiv paper from Zixuan Fu, Bingxiang He and collaborators titled 'Rethinking On-Policy Distillation of Large Language Models II: One Training Example.'

Bump the query count from one to sixteen semantically distinct queries and coverage jumps to 98.9%, matching full-dataset performance. Most of that coverage accumulates within the first 100 training steps. The authors report that the single-query run captures 'most of full-data OPD's gain across task domains and model families.'

Their framing for the pattern is blunt: 'OPD is therefore data-overfed but algorithm-starved.' Student models absorb supervision increasingly slowly regardless of dataset size, which the authors read as a computational bottleneck rather than a data one. The result extends to multi-teacher OPD, where 16 diverse queries per domain again replicate full-data performance.

The retrieved abstract summary does not identify the specific benchmark suites, student sizes, or teacher models behind the 71.5% and 98.9% numbers.