WebArena-Pro: Coding Agents Score 62.3% on Web Tasks; Generic VLMs Score 30.3%

Found first: a primary source the press has not covered yet.

A 22-author team has published WebArena-Pro, a 300-task benchmark for web agents running across 20 self-hosted applications in six domain categories. The top coding-agent systems scored 62.3%; the best generic vision-language models scored 30.3%. The paper is accepted to the NeurIPS 2026 Education and Dissemination Track.

What the source says

Authors include Imene Kerboua, Xing Han Lù, Alexandre Lacoste, Huan Sun, and Gunhee Kim, among 17 others. The benchmark adds audio and video as first-class observation modalities alongside text and vision, with dedicated extraction actions for each. Each task runs in isolation to enable reproducible parallel evaluation; tasks are human-authored with LLM-assisted triage and human validation. Two systems tied at the top: Claude Code paired with Claude Opus 5.5, and Codex paired with GPT-6.1 Sol, both at 62.3%. OpenCode with Kimi 3.1 Pro reached 60.7%. Gemini 3.1 Pro Omni and Gemma4 31B each scored 30.3%.

Why it matters

Prior web-agent benchmarks treated audio and video as edge cases or omitted them entirely; WebArena-Pro makes them required. The gap between coding-agent architectures and generic VLMs is roughly two to one across the same 300 tasks, which gives teams a concrete, reproducible signal about architecture choice rather than a claim tied to a single model comparison. The self-hosted setup removes live third-party service dependencies, so the numbers can be reproduced and compared across runs. Teams currently choosing between coding-agent and VLM-native approaches now have structured multimodal evidence to inform that decision.