Minima Quantizes Qwen3.8-27B GDN Layers to NVFP4, Matches BF16

Every public 4-bit build of Qwen3.8-27B protected the same thing. The 48 Gated DeltaNet layers, and especially the two gate projections that control the forget rate α and the write strength β, stayed at 8- or 16-bit, on the intuition that quantization errors in a recurrence would compound over long contexts. A new paper from the Minima team argues that intuition is backwards. Alibaba's own Qwen3.8-Max-0902 snapshot landed on our tracker the same week.

The team quantized all 496 linear layers of the 27B hybrid (240 GDN, 64 attention, 192 MLP) to NVFP4 W4A4, GDN included. The resulting checkpoint matches BF16 within seed noise on MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench and RULER at 64K, with a five-task average 0.52 points off BF16. On AIME'25 it scored 26/30 on all four seeds, matching BF16 exactly. The file weighs 17.5 GiB, 2.9× smaller than BF16 in VRAM, and prefill TTFT falls from 6.90s to 4.03s on a 32K prompt.

Then the paper explains itself. In a per-projection sensitivity replay, the two supposedly fragile gate projections were the least sensitive tensors in the entire layer: fully quantizing a and b moved the layer output by 2.1% and 2.6%, "the two smallest effects, even though their own GEMM errors are 11.0% and 8.5%." The log-space softplus/exponential and sigmoid parameterizations "compress a ~11% GEMM error to a ~2% output error." A 32K FP32 lockstep experiment then shows state error hitting a flat plateau (12.96% at token 256, 12.31% at token 32,768) rather than accumulating, because "every write overwrites the state along the current key direction, so old errors are deleted key by key as new tokens arrive." Context washes the 4-bit cost out; it does not build.

The paper also flags a serving bug that has not yet bitten a shipped recipe but will bite the next one. llm-compressor calibrates one global scale per linear module; vLLM fuses adjacent GDN projections into single GEMMs, taking the maximum of the paired scales without rescaling the locals. In Minima's checkpoint the paired scales differed by 1.82× (qkv/z) and 2.75× (b/a) in every one of the 48 layers, silently mis-scaling the decay and write gates. The corrupted checkpoint is deceptively plausible: reasoning drops moderately (AIME 80.8 vs. 86.7 repaired) while long-context perplexity gets better than BF16, because "a broken forget gate makes the state hold everything, which happens to help next-token prediction on WikiText." The fix rewrites each fused group to the shared global scale and folds the ratio into the per-block E4M3 scales.