HEELO primitives beat Claude Opus 5 harness on Terminal-Bench 4.0

On Terminal-Bench 4.0, a new agent framework called HEELO reports 55.5% task resolution using a GPT-5.6 Sol backbone with "Dev-Primitives," ahead of Claude Opus 5 running Claude Code at 51.8%. A frontier model with its own in-house harness, outscored by a cheaper backbone wrapped in a different scaffolding layer.

The arxiv preprint describes HEELO, short for Harness Engineering around Evolving Agent-Native Primitives, as a library of reusable units, each pairing a resource with "a resident LLM that applies it in context and communicates with other primitives." For each task, the harness "composes the primitives a task needs into a task-specific graph," verifies the output against environment evidence, and revises the primitives a failure implicates.

The abstract lists gains across four families. Dev-Primitives raise Terminal-Bench 4.0 resolution "from 37.3% under Codex to 55.5% on the same backbone." Harness Primitives lift Terminal-Bench 2.0 Pass@1 from 60.0% to 72.2%. Tool Primitives push ToolBench from 55.8% to 75.1%. Navigation Primitives take GAIA-Text-103 accuracy from 45.6% to 58.3%.

On the authors' 46-task heelo.dev platform, Claude Opus 5.5 with Claude Code and GPT-6.1 Sol with Codex each finish 9 tasks alone. Connected to HEELO through an MCP server, they finish 39 and 35 respectively, "with 1.9× and 2.3× less time and 1.55× lower cost per completed task." The 46 tasks were assembled by the authors, and the paper is a single preprint; third-party runs on tasks the authors did not design would settle whether the gap holds.