Gemma-4 12B tops 31B on AIME via KV-cache grafting trick

The paper landed as a single-author July 2026 arXiv preprint, and the numbers are big enough that they deserve careful reading before anyone gets excited. Sietse Schelpe describes a method he calls byte-exact KV-cache grafting: instead of retraining a small model or bolting on retrieval, you take verified solutions computed once and store them as precise key-value cache states that can be restored into a fresh inference run without touching weights. According to the arXiv preprint, the restore is byte-for-byte identical to a fresh computation, checked with SHA-256 equality and zero KL divergence.

On AIME 2025, the reported jump for Gemma-4-12B is 80.0% up to 93.3% with grafted verified solutions, above the 31B sibling's 89.2% and well above the 12B's own 77.5% anchor. The efficiency claim is the more striking one. Eight problems the model previously could not solve were answered using 61 decode tokens against a 401,026-token budget, which the paper frames as roughly 6,574x fewer tokens and about 8,700x less energy. The author also reports the effective context stretching from 32,768 to 2,854,766 tokens without additional GPU memory, with input and output hashes committed for independent verification.

The honest caveat is that this is a single-author preprint, not peer reviewed, and AIME 2025 is exactly the sort of benchmark where pre-computing verified solutions once and replaying them is easiest to look good on. Committing SHA-256 hashes for outside checks is the right move, but the harder questions are whether grafting carries over to problems the cache has not seen, whether the trick survives outside the Gemma-4 family, and how much upstream compute went into producing those verified solutions in the first place. The write-up does not really price that upstream cost.

If the technique holds up in outside hands, the more interesting shift is who benefits. A small model on constrained hardware getting hard-reasoning capability from a library of pre-verified traces rather than from bigger weights is a different economic curve from fine-tuning, RAG or distillation, and worth tracking as replications land.