LHTB benchmark: top frontier agent clears 15% of terminal tasks

A team of researchers has put a concrete number on a question the agent industry has mostly been ducking: how do frontier models actually do when a task takes hours instead of seconds? The answer, per the paper on arXiv, is not great. Even the strongest tested model clears just 15.2% of tasks at a partial-reward threshold, and the average across the 15 frontier models evaluated sits at 4.3%. At the harder perfect-reward bar, the mean drops to 1.7%.

Long-Horizon-Terminal-Bench, or LHTB, is 46 tasks across nine domains, spanning experiment reproduction, software engineering, multimodal analysis, interactive games, and scientific computing. What makes it different from the benchmarks that agent vendors like to quote is scale: a single task run averages 85.3 minutes of execution, roughly 231 episodes, and 9.9M tokens. The grading is dense rather than a pass or fail on a final artifact, so partial credit is available for making real progress even if the agent never finishes.

Why this matters if you are not writing agent papers: this is the first evaluation I have seen that puts a hard ceiling on the gap between autonomous-agent marketing and what today's models actually deliver on multi-hour work. A 15.2% top score with partial credit is a very different story from the near-solved coding benchmarks that scaffold most enterprise pitches. It suggests the honest use case for these systems is still bounded, supervised runs, not fire-and-forget research assistants.

The caveats worth naming: the retrieved material does not identify which of the 15 frontier models hit the top score and which sat near the floor, so vendor-by-vendor comparisons are not possible from the abstract. It also does not go deep on which agent scaffolds were used around each model, and scaffold choice is often where a lot of the variance actually lives in agent evaluations. And 46 tasks, even hard ones, is a small sample from which to generalize to production workloads.

Still, the direction is the useful part. Dense reward grading on genuinely long tasks gives researchers a training signal to work against, and gives buyers a sanity-check ruler for the next round of autonomous engineer or scientific-computing agent pitches.