Found first: a primary source the press has not covered yet.
A preprint submitted August 14, 2026 introduces S²VOPD (Self-Supervised Visual On-Policy Distillation), a distillation method that lifts a 4B multimodal model's average score across six fine-grained visual perception benchmarks from 70.7% to 77.4%, surpassing Qwen3-VL (235B) and GPT-5.4, with no privileged data or ground-truth annotations required. The paper is authored by Yijiang Li, Yijun Liang, Yunjie Tian, Bingyang Wang, Ke Zhang, Zhenfei Yin, Di Fu, Philip Torr, and Nuno Vasconcelos.
What the source says
Rather than giving the teacher extra information, S²VOPD restricts the student: the teacher generates predictions from the original image while the student learns from a heavily augmented version of the same image. This asymmetry is the operative factor. All four augmentation families the authors tested improved performance; symmetric self-distillation, where both teacher and student see augmented views, degraded it. Applied to Qwen3.5-4B, the method recovers 96% of the improvement achieved by distillation methods that do use privileged supervision, while requiring no additional annotations or data pipeline.
Why it matters
The standard intuition in multimodal scaling is that closing the gap to larger models requires either more data, more compute, or privileged training signals. This paper achieves a 6.7 percentage point gain by doing the opposite: subtracting information from the student. A 4B model reaching scores above a 235B open-source model through an annotation-free training change has direct implications for practitioners working under inference-cost or deployment constraints. The finding that asymmetry alone drives the gain, rather than any specific augmentation type, suggests the principle may transfer broadly across architectures and tasks.