Nvidia's Nemotron-Labs-Diffusion unifies AR and diffusion modes

A new paper from Nvidia introduces a language model family called Nemotron-Labs-Diffusion that trains a single model to decode text in three different ways and lets you switch between them at inference. The paper is on the Hugging Face papers hub and the weights (3B, 8B, and 14B, with base, instruct, and vision-language variants) are posted to the Nvidia collection on Hugging Face.

The novelty is the training objective. Rather than picking either autoregressive next-token prediction or diffusion-style parallel denoising, the authors train both together with a weighted loss, and their ablation study finds the two are complementary rather than competing. Adding the AR loss to a diffusion baseline was the single biggest lift in their table, a 7.48% jump in average benchmark accuracy, and adding the diffusion loss on top of AR training leaves AR-mode accuracy essentially unchanged, with reported 0.14% and 0.43% gains for base and instruct models.

The headline throughput claim is that Nemotron-Labs-Diffusion-8B decodes 6x more tokens per forward pass than Qwen3-8B at comparable accuracy, which the authors say translates into 4x higher throughput on SPEED-Bench measured with SGLang on a GB200 GPU. Their own tables show the 8B model at 2.57x tokens per forward in pure diffusion mode, rising to 5.99x under a LoRA-tuned linear self-speculation scheme where diffusion drafts and AR verifies inside the same weights. A quadratic self-speculation variant reaches 6.38x, though the authors note its real-device efficiency currently trails the linear one because the required attention masks run on less optimized FlexAttention kernels.

The honest caveat sits in the paper's own speed-of-light analysis. Current confidence-based diffusion sampling achieves only around 3x tokens per forward at comparable accuracy, while the oracle upper bound is 7.60x, a large chunk of parallelism that future samplers still have to reach. The instruct model also regressed on IFEval by 3.01% and on HumanEval by 2.44% when the diffusion loss was added, suggesting strict instruction-following is where the joint objective still costs something. What the paper does not give you is cost or latency numbers on hardware people actually own, or head-to-head figures against closed-source frontier models; the comparisons are against open baselines like Qwen3, Ministral3, LLaDA, Dream, and SDAR.

If even part of that speed-of-light ceiling is reachable with better samplers, this becomes a template rather than one release: an AR checkpoint that quietly serves as its own draft model at low concurrency and reverts to plain AR at high concurrency, without a second model or a new inference pipeline.