WebArena-Pro (NeurIPS 2026): Coding Agents Score 62.3% on Web Tasks, Generic VLMs 30.3%

Found first: a primary source the press has not covered yet.

A new benchmark called WebArena-Pro, accepted at NeurIPS 2026's Education and Dissemination Track, puts coding-agent architectures at 62.3% on 300 web tasks while the best generic vision-language models reach 30.3%. The paper, by Kerboua et al., is the first web-agent benchmark to treat audio and video as core observation modalities alongside text and screenshots.

What the source says

WebArena-Pro covers 300 tasks across 20 self-hosted web applications in six domain categories, with dedicated extraction actions for audio and video content, not just text and vision. Tasks run in full isolation, allowing reproducible parallel evaluation. At the top of the leaderboard, Codex with GPT-6.1 Sol and Claude Code with Claude Opus 5.5 both scored 62.3%; OpenCode with Kimi 3.1 Pro reached 60.7%. Generic VLMs scored substantially lower: Gemini 3.1 Pro Omni and Gemma4 31B both reached 30.3%. Tasks were authored through a dedicated annotator interface, filtered by LLM-assisted triage, and validated by humans before release.

Why it matters

The roughly 2x gap between coding-agent architectures and generic VLMs is a concrete, actionable signal: on this benchmark, the agent loop contributes as much to task success as the underlying model's raw capability. Prior web-agent benchmarks relied on text and screenshots; requiring audio and video extraction tests a broader capability set that reflects what real applications actually serve. Task isolation directly addresses the shared-state contamination problem that made earlier web-agent benchmarks difficult to reproduce. The leaderboard numbers are specific enough that teams can use them to make build-vs-buy decisions on agent architecture today.