Alibaba Qwen paper isolates a CFG bug in diffusion distillation

The interesting thing in a new paper out of Alibaba's Qwen Business Unit is not a new model, it is a diagnosis. In Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation, the authors argue that the standard way people distill diffusion models under classifier-free guidance is quietly under-identified, and that a well-behaved training loss can hide a real failure mode.

The setup is familiar. Modern diffusion systems denoise with a composed velocity that mixes a positive prediction (under the target text condition) and a negative prediction (under a null or negative condition), weighted by a guidance scale chosen at inference. On-policy distillation asks the student to match the teacher along the student's own trajectories, and the natural extension is to match the CFG-composed guided velocity. The problem the authors identify is that this objective constrains only the composition, so positive-branch and negative-branch errors can compensate for each other and the loss will still look fine.

Under shared negative conditioning, in a text-rendering setup using an SD3.5-Medium-based OCR teacher, both branch errors drop together and naive matching is benign. Under privileged negative conditioning, in a reference-conditioned setup on FLUX-2-klein-base-4B where the teacher's negative branch retains a reference image the student never sees, the authors report that naive matching reduces the positive-branch error while increasing the negative-branch error. They term this failure mode Negative Branch Asymmetry. The tell is guidance-scale sensitivity: the distilled student can track the teacher at the training guidance scale yet drift when the inference scale changes, most visibly at γ=1.

Their fix, Positive-Direction Matching, separately supervises the positive prediction and the CFG conditional direction v⁺ minus v⁻, so zero loss forces both branch errors to zero. Applied to dense-to-sparse video control on Wan-VACE across pose, depth, and scribble modalities, they report that naive matching degrades sharply as inference γ departs from the training value, while PDM and an Independent Branch Matching baseline remain stable.

The honest caveat is scope: two image case studies plus one video application, and the retrieved paper does not give a lot of headline quality numbers to wave around, nor training-cost budgets. What the reporting also does not settle is how branch-aware OPD behaves on newer diffusion systems that move away from CFG. The forward-looking point is more useful for practitioners: if you distill CFG-based diffusion models and only ever evaluate at the training guidance scale, the bug the paper names may already be in your ship queue.