SoloQ cuts diffusion LLM memory 2.61x without calibration data

SoloQ, a new quantization framework for diffusion large language models, cuts peak memory by up to 2.61x and accelerates end-to-end inference by up to 2.24x at 4-bit precision without any calibration data, according to a new arxiv preprint.

The technique maps weights and activations into what the authors call a 'normalized rotated basis with a predictable marginal distribution,' combining a 'structured K-RPBH rotation with a lightweight rescaling correction.' That matters because standard post-training quantization leans on calibration runs that generalize poorly for dLLMs, where 'activation distributions shifting across masking states and denoising steps' move the target between forward passes. For block-diffusion models, SoloQ adds a commit-time KV-cache quantization step that compresses persistent states 'without perturbing the actively denoised block.'

The authors evaluate on four diffusion LLMs: LLaDA and Dream in the full-sequence setting, plus Fast-dLLM v2 and Nemotron-Labs-Diffusion for block-diffusion. The paper claims SoloQ retains accuracy at 4-bit and 'outperforms calibration-based baselines on knowledge- and reasoning-intensive benchmarks,' with the headline 2.61x memory figure and 2.24x speedup specifically tied to NVFP4. No per-benchmark accuracy numbers appear in the abstract, so the size of any quality gap at 4-bit is left to the full paper.