Microsoft HCU tops OSWorld 2.0 with three agent channels

Microsoft researchers, with collaborators at Fudan and Tsinghua, released a paper arguing that screenshots are the wrong default for computer-use agents. Their system, Hybrid Computer Use (HCU), gives the model three ways to touch a machine instead of one: run code, read the accessibility tree, or click the screen like a person. On OSWorld 2.0 it posts the highest scores in the comparison table, 72.37% Partial and 32.40% Binary driven by GPT-6 Astra, and 72.06% Partial with 35.19% Binary driven by Claude Opus 5.

The pitch, in the authors' own words: "interfaces centered on screenshots and mouse-and-keyboard actions can impose a human-interaction prior, requiring repeated screen interpretation while leaving the computer's semantic and programmatic capabilities underexposed." The paper on arXiv argues that exposing those capabilities, and letting the model pick which lane to use, matters more than sharper vision.

The efficiency number is the striking one. HCU "outperforms CUA Driver by 3.55 points in Partial and 0.90 in Binary with 29.81% fewer tool calls at nearly identical Cost," the paper reports. On Windows Agent Arena with GPT-5.4, HCU reaches 80.86% against CUA Driver's 71.89%; on WeaveBench, 53.25% versus 46.37%.

A supporting "skill", essentially a prompt that tells the agent how to pick a channel and format arguments, does heavy lifting. The authors report it cuts "the proportion of AX and GUI calls rejected for malformed arguments from 17.37% to 1.36%." Different models allocated work across the channels differently.

Code is on Microsoft's GitHub. The abstract names no production deployment and does not break out per-app failure modes where the accessibility tree is sparse.