Ant Group's LLaDA MoE v2 nears Qwen3 on 65% of the tokens

Diffusion for text has spent the last two years in the interesting-demo bucket, a step behind the autoregressive frontier and mostly stuck at small scale. A paper posted this week to arXiv is the first release I've seen that pushes back on that framing at real scale: LLaDA MoE v2, a mixture-of-experts diffusion language model with 30 billion total parameters and 3 billion active, trained from scratch.

The headline number is efficiency, not raw score. The authors report the model was trained on 23.5 trillion tokens, roughly 65% of what went into Qwen3 in the same weight class, and that it "approaches Qwen3 on several knowledge, reasoning, and coding benchmarks." After supervised fine-tuning it "outperforms SDAR Chat on seven of eight reasoning and coding benchmarks" while staying close to Qwen3. Nothing in the abstract says diffusion overtakes autoregression, but the token-efficiency framing is where the field's attention has been drifting.

The other useful piece is architectural. The paper argues for a specific set of scaling laws and design choices for MoE diffusion LLMs rather than borrowing the autoregressive playbook wholesale, describing its contribution as "practical scaling laws and design principles for MoE dLLMs." If those recipes hold up under third-party replication, the practical implication is that the next generation of diffusion LMs could keep closing the gap without waiting for another compute-scale cycle.

The honest caveat is that this is a fresh preprint from the authoring team's own evaluations, and "approaches" is doing quiet work in the abstract, so treat the specifics as reported rather than settled. What the writeup doesn't hand you is per-benchmark scores, inference-cost comparisons at deployment, or whether the SFT recipe was matched fairly to Qwen3's post-training pipeline.

For anyone tracking non-autoregressive architectures, the takeaway is that diffusion at 30B-A3B is now close enough to make the "should we bet on this?" conversation less speculative and more of a real planning question. That is where the interesting work over the next few months is going to happen.