Flash-dLLM posts 11x HumanEval, 5.1x GSM8K dLLM speedups
TL;DR
- Flash-dLLM reports 5.1× and 11.0× training-free speedups over the Elastic-Cache baseline on GSM8K and HumanEval respectively.
- The framework identifies GPU memory I/O as the dominant bottleneck in KV-cache-enabled diffusion LLM inference.
- In decoding, the diffusion LLM itself acts as both drafter and verifier, removing the auxiliary drafter model.
A new preprint reports a training-free trick that "achieves 5.1× and 11.0× speedups over prior strongest baseline Elastic-Cache on GSM8K and HumanEval, respectively" for diffusion language models. Authors Quan Nguyen-Tri, Mukul Ranjan, and Zhiqiang Shen posted the Flash-dLLM paper on arXiv, framing the problem as I/O rather than compute.
The abstract identifies "GPU memory I/O as a dominant bottleneck in KV-cache-enabled dLLM inference" and addresses it with an I/O-aware fused KV-cache kernel that trims redundant memory movement. On top of that sits a draft-and-verify decoding loop in which "the dLLM itself serves as both drafter and verifier without requiring an auxiliary model."
The claim is that the combination holds generation quality steady while scaling to longer sequences and larger batches, on mathematical-reasoning and code-generation benchmarks.
The abstract does not name which diffusion LLM checkpoints were tested, does not publish per-benchmark quality numbers, and reports the headline speedups only against one baseline. It is an unrefereed arXiv preprint.
Originally reported by paper
Read the original article →Original headline: Flash-dLLM Delivers 11x Speedup on Diffusion LLM Inference With No Retraining