Yandex: Register Tokens Help Pixel-Space DiTs More Than Latent

A quieter paper out of Yandex Research on arXiv is worth pulling up if you are following the shift from latent-space to pixel-space diffusion models, because it tries to explain an architectural puzzle rather than chase a leaderboard.

The question the authors ask is whether register tokens, the small add-on tokens that fix high-norm patch-token outliers in Vision Transformers, matter for Diffusion Transformers too. What they find is a little strange. DiTs, according to the paper, do not exhibit the patch-token outliers ViTs suffer from, yet they still benefit from registers, and the benefit is noticeably larger in pixel-space DiTs than in latent-space DiTs. Their reading is that register tokens produce cleaner feature maps at high noise levels, which appears to matter more when the model is denoising directly in pixels than when it is working over a compressed latent.

Why this matters if you are not writing generation research: the field has been quietly drifting back toward pixel-space training as compute cheapens and latent autoencoders show their compromises. The paper's other observation is that recent pixel-space DiT architectures implicitly incorporate register-like mechanisms, which the authors argue may partially account for their strong empirical performance. They bundle the intuition into a technique they call Register Guidance, described as amplifying the contribution of register tokens responsible for improving visual structure and coherence.

The honest caveat is that the abstract does not give quantitative benchmark numbers, so you are being asked to take the mechanism story on the strength of the analysis, not on a headline FID delta. What the paper's abstract does not yet tell you is the compute overhead of Register Guidance at inference, or how far the effect scales beyond the authors' experimental settings.

Still, if you are building or scaling a pixel-space image or video generator, this reads as a cheap inference-time knob to test rather than a retrain, and a plausible piece of the puzzle for why the pixel-space direction has been quietly winning.