LLM-as-a-Verifier framework posts 86.5% on Terminal-Bench V2

The interesting move in a new arXiv paper posted July 6 by Jacky Kwok, Chelsea Finn, Marco Pavone, Ion Stoica, Azalia Mirhoseini and colleagues is not the benchmark table, it is the framing. Their LLM-as-a-Verifier paper treats verification, the act of grading an answer, as its own scaling axis for LLM improvement. That is a claim about where headroom now lives.

The mechanism is worth reading closely. Instead of asking a judge model to emit a discrete score, the framework "computes the expectation over the distribution of scoring token logits to generate continuous scores." Those continuous scores are then scaled across three levers the authors call out: score granularity, repeated evaluation, and criteria decomposition. The claim is that finer scoring granularity improves separation between positive and negative solutions, while repeated evaluation and decomposition cut variance.

The benchmark numbers the paper reports are 86.5% on Terminal-Bench V2, 78.2% on SWE-Bench Verified, 87.4% on RoboRewardBench and 73.3% on MedAgentBench. The authors also describe integrating the verifier with Claude Code for developer system monitoring, and using the fine-grained signal as dense feedback for reinforcement learning on robotics and mathematical reasoning tasks trained with SAC and GRPO.

The honest caveat is the usual one for a fresh preprint. This is a single paper making a broad framing claim across four very different domains, and the abstract page does not itself let you audit the eval setups, the baselines, or whether the same three levers help every benchmark equally. Take the specifics as reported, not settled, until independent groups run the same evals or the code lands.

Where this could actually matter is downstream. If continuous, cheap verification really does become a reliable knob, the people building agent stacks and RL pipelines get a much denser signal to optimize against than "did the task pass, yes or no," and that changes what it costs to train the next generation of agents.