Found first: a primary source the press has not covered yet.
Minsoo Cheong and Sungjoo Yoo introduce LoopVQ, a KV cache quantization method that reaches 0.5 bits per element and outperforms existing KV cache vector quantization approaches across looped transformer models. Implemented in SGLang, it nearly halves decode latency at a 32K-token context and raises peak serving throughput by up to 6.1x on a 2.6B Ouro model versus BF16.
What the source says
LoopVQ codes channels from multiple consecutive loop steps jointly into a shared codebook, then fine-tunes those codebooks to match the full-precision model's output. At 0.5 bits per element, it outperforms existing KV cache vector quantization methods on general language modeling, long chain-of-thought reasoning, and long-context retrieval, evaluated on Ouro (1.4B and 2.6B) and Huginn models. Combined with TaSQ, the method reaches 0.25 bpe while retaining much of BF16 accuracy. The SGLang integration nearly halves decode latency at 32K context and raises peak serving throughput by up to 4.3x on the 1.4B Ouro model and up to 6.1x on the 2.6B Ouro model. No institutional affiliation is listed in the abstract.
Why it matters
Looped transformers share weights across multiple forward passes, creating KV caches with cross-step redundancy that standard per-layer quantizers ignore. LoopVQ's joint coding across loop steps is what enables sub-1-bit compression on these architectures; the approach does not transfer to ordinary transformers. For practitioners serving looped models at long contexts, the throughput figures are practically significant: up to 6.1x on the 2.6B Ouro model versus BF16. At 32K context, the nearly halved decode latency indicates the gains are sharpest where KV cache size is the dominant serving bottleneck.