StateAct Cuts Computer-Use Agent Cost 9x on OSWorld 2.0

Most computer-use agents work the way a person would if you handed them a keyboard and covered the terminal, taking a screenshot, reasoning about the pixels, deciding on a click, then doing it again. A new arXiv paper introducing StateAct argues that framing is expensive by design, because a screenshot is only a lossy rendering of the underlying program state the agent could read directly.

The setup is a code-first main agent with a small GUI subagent bolted on for the moments when there really is no other way in. On OSWorld 2.0 with Claude Opus 4.8, binary success climbed from 20.6% to 26.9% and partial success moved from 54.8% to 61.6%, at roughly one-ninth the per-task cost of a screenshot-only baseline running the same model. The GUI subagent was invoked in only 28 of 108 tasks and handled about 1.1% of the main agent's steps. The rest of the work happened by inspecting and mutating files, application backends, and the DOM through code, with an independent verification gate checking outcomes against program state rather than pixels.

If those numbers hold up in independent runs, the interesting implication is not the six-point accuracy bump, it is the cost curve. Long-horizon agent products have been stuck with the same economics for a year, because every step multiplies token count when you keep feeding fat screenshots back to a frontier model. A design that only touches pixels when it has to changes the unit economics of running a computer-use agent in production.

The honest caveat is that this is a preprint from a single team, on one benchmark, with one model. The paper does not put a dollar or wall-clock figure on either configuration, does not report on closed GUI-only apps where the code path simply is not available, and does not say whether the same ratio holds on cheaper backbones than Claude Opus 4.8. The verification gate is a nice touch but it can only catch failures the state check itself was written to notice.

The direction worth watching is whether 'read the state, do not look at the screen' generalizes. If it does, the practical winners are the RPA and browser-agent vendors who can rewrite their runtimes around it before the incumbents notice the margin has moved.