Berkeley, Reve corrector step curbs diffusion exposure bias

The denoiser in a diffusion model is trained on real images with noise, then at inference asked to clean up its own earlier guesses, something it has never seen. The paper "Correction Training for Diffusion Models" by Donghoon Ahn, Runqian Wang and Alexei A. Efros (UC Berkeley) with Taesung Park and Yash Belhe (Reve) calls this an exposure bias and bolts a corrector step onto the sampler.

"Iterative sampling in diffusion models can lead to error accumulation due to exposure bias," the abstract reads. "The denoiser is trained on real images interpolated with noise, but at inference its inputs are constructed from its own imperfect predictions from the earlier steps, something it has never seen at training time." The corrector takes the base model's prediction as input, trains against the same ground-truth target, and at sampling time refines predictions iteratively, "preventing early artifacts from persisting into the final sample."

The authors report evaluations across DiT, SiT and JiT, with the method "competitive with classifier-free guidance" and particularly helpful for unconditional generation, "where producing high-quality samples without guidance has been particularly challenging." In an ablation on LightningDiT at ImageNet 256×256, a 14-layer denoiser paired with a 14-layer corrector applied three times reached FID 5.66, versus FID 8.93 for a 28-layer denoiser under an identical compute budget.

The paper claims the inference-time overhead can be amortized by distilling the corrected predictions back into the base denoiser.