Grouped Value Attention cuts KV cache 45-47% vs GQA at 350M
TL;DR
- GVA reports cutting persistent KV cache scalars by roughly 45–47% versus a matched GQA baseline in a September 8 arxiv preprint.
- At 350M parameters on 30B FineWeb-Edu tokens, GVA averages 44.18 across five tasks against 44.36 for GQA and 43.88 for MLA.
- Content keys are reconstructed on the fly via a learned linear map absorbed into queries, with a decoupled RoPE channel carrying positions.
A new attention variant called Grouped Value Attention drops the persistent KV cache by roughly 45 to 47 percent relative to a matched Grouped Query Attention baseline, according to authors Vishesh Tripathi, Abhay Kumar and Ramsha Khan in a September 8 arxiv preprint.
The mechanism is compact. GVA "stores grouped values and reconstructs content keys with a learned linear map," and the paper says that map can be absorbed into queries at inference time, so content keys never need to materialize in cache. A separate decoupled RoPE channel keeps positional information alive through cached positional keys.
Accuracy at 350M parameters, trained on 30B FineWeb-Edu tokens, lands close to the GQA baseline: 44.18 average across five evaluated tasks for GVA with a 16-dimensional positional channel, against 44.36 for GQA and 43.88 for MLA. The authors describe the result as "near-GQA benchmark accuracy with a more compact cache representation."
Custom decoding kernels have been developed but end-to-end inference performance is still under evaluation, and the authors say an open-source release is planned pending that work. The evaluation stops at 350M, so whether the near-parity claim survives at frontier scale is not addressed in the abstract.
Originally reported by paper
Read the original article →Original headline: Grouped Value Attention Eliminates Key Cache Entirely, Cuts KV Memory 45–47% vs GQA at Near-Parity Accuracy