Diffusion LMs: Decoding Speed Beats Majority Voting by 4.2pt

Wrong math answers cost more to decode.

"On the same math problem, wrong answers take more passes per token than correct ones, and weighting candidate answers by decoding speed beats majority voting by up to 4.2 points at no extra cost," the authors report in a preprint posted to arXiv on October 19, 2026. The study, by Mohsen Fayyaz, Ali Modarressi, Daniel Israel, Lucas Bandarkar, and Nanyun Peng, analyzes two diffusion language models across 10 tasks.

Diffusion LMs decide, mid-generation, how many forward passes to spend, and the paper's claim is that this variable cost is informative. In DiffusionGemma, "generating a token for story writing requires more than three times as many forward passes as for grammar correction." Two output properties explain most of the gap: how much of the output is new relative to the input, and how rare its words are within the task.

The pattern holds inside a single response. "The final answer is cheap when it repeats the model's reasoning trace and costly when it adds something new."

Multilingual generation stacks the penalty: highly tokenized languages pay more tokens and more passes per token on top. The 4.2-point figure is an upper bound ("up to"); per-task breakdowns are not in the abstract.