AdaptiveSpec: up to 56% over EAGLE-3, no training needed

A new arxiv preprint claims up to 56% higher throughput than EAGLE-3 on speculative decoding, using a training-free method the authors call AdaptiveSpec. Two AI experts in our Who's Who directory have shared the paper.

The pitch is to stop holding two decisions fixed. Tree-attention drafters like EAGLE-3 typically keep a strict token-match verification rule and a static draft-tree shape; AdaptiveSpec adapts both from signals produced during decoding. 'A per-step margin rule promotes a mismatched draft-proposed token when the ratio of the target's probability on the drafted token to its top-1 probability exceeds a threshold,' the authors — Oszkár Urbán, Young D. Kwon, Stylianos I. Venieris, and Cecilia Mascolo — write, adding that the rule has 'no dependence on draft length or underlying drafter architecture.'

Alongside that, a per-step tree policy 'adjusts the draft tree's depth, width, and node count directly from a fused signal of draft top-1 confidence and a rolling acceptance history capturing recent draft-target agreement.' The two adaptations, the paper says, 'operate on orthogonal axes and compound in effect.'

Implemented on the SGLang production-grade serving engine, AdaptiveSpec is benchmarked against three 8B-class targets — DeepSeek-R1-Distill-Llama-8B, Llama-3.1-8B-Instruct, and Qwen3-8B — across GSM8K, MATH-500, and HumanEval, 'recovering 93% to fully lossless task accuracy.' That framing is honest about what this is: an explicit lossy scheme with an accuracy floor, in exchange for the throughput win.

The abstract does not break out per-benchmark accuracy or per-model throughput; the top-line 56% is the ceiling, not the average.