S²VOPD lifts Qwen3.5-4B above Qwen3-VL 235B and GPT-5.4

A 4B multimodal model climbs from 70.7% to 77.4% across six fine-grained perception benchmarks after a distillation setup that adds nothing to the teacher and instead strips information from the student. The result, posted to arXiv on 14 August by Yijiang Li and eight co-authors, puts the tuned Qwen3.5-4B "above all open-source models compared, up to Qwen3-VL at 235B, and surpasses GPT-5.4."

The method, called Self-Supervised Visual On-Policy Distillation, or S²VOPD, "distills the teacher's distribution conditioned on the original image on-policy into the student distribution conditioned on a strongly augmented view of the same image." Same weights on both sides; the asymmetry is only in what each view sees. The authors frame the move as an inversion: "Rather than adding privileged information to the teacher, we subtract information from the student."

Three findings shape the recipe. All four augmentation families the authors test improve results, while symmetric self-distillation actively degrades performance. Performance peaks at moderate augmentation strength; push too far and "augmentations that completely remove the question-relevant evidence can induce large but uninformative discrepancies." Held to the same training data, the technique "recovers 96% of the improvement achieved by methods with privileged information," leaving a residual gap the abstract does not close.

Only the aggregate figure across the six benchmarks is reported. The abstract lists no per-benchmark numbers and confines its claims to fine-grained perception.