Gambit Beam Search Cuts Reasoning Tokens 68.5%, Adds 6.7pp

Test-time compute is the lever everyone in the reasoning-model race is pulling right now, and a new paper on arXiv from Lijie Yang, Hongyin Luo, Jiawei Zhao, Tri Dao and Ravi Netravali argues the useful question is no longer how much to spend but where to spend it. Their method, Gambit, is described as an inference algorithm that executes thought-level beam search, periodically pruning unpromising trajectories and immediately branching from high-quality prefixes, guided by what the authors call a light-weight scorer probing hidden states.

The headline numbers, under identical hardware constraints, are up to a 6.7 percentage-point absolute accuracy gain on HMMT-24 and 3.3 points on AIME-25 over pruning baselines, together with more than 2x higher throughput on trace completion and up to a 68.5% reduction in total token consumption relative to standard parallel sampling. The framing sets Gambit against two dominant paradigms the authors say fail in opposite directions: parallel sampling treats traces independently and induces severe memory bottlenecks, while subtractive pruning starves the accelerator and does not shift the output distribution enough.

For anyone paying an inference bill for reasoning workloads, the interesting claim is that you can, in principle, get more accuracy and use fewer tokens at the same time on the same GPUs. That combination is unusual in this literature, where efficiency work tends to concede a bit of quality for cost.

A few things the abstract does not settle. The accuracy gains are stated only against pruning baselines, not against best-of-N with a trained verifier or other stronger comparators. Both benchmarks are competition-style math, and the abstract does not list which base reasoning models were used or how the hidden-state scorer would transfer across model families. The 68.5% token cut is measured against standard parallel sampling, not against whichever inference stack a given team already runs in production.

If the results generalize beyond math and drop cleanly into serving stacks like vLLM or SGLang, this is the kind of technique inference vendors and cost-sensitive open-weight users will want on the eval bench well before the next reasoning-model refresh.