VisSearch-Bench: Best Multimodal Agent Solves 26.47% of Tasks

Multimodal agents equipped with web and image tools fully solve just 26.47% of tasks on VisSearch-Bench, a new evaluation that pushes beyond treating images as entity identifiers. That is the top score among 16 systems the authors ran.

The benchmark, posted to arXiv on 3 November 2026, contains 204 human-written tasks across seven main domains and 25 subdomains, split into five task types: Set-wise Visual Search, Sequential Visual Search, Hybrid Wide-Deep Search, Competitive Visual Disambiguation and Visual-Structure-Guided Search. The authors argue that prior search benchmarks "use images mainly to identify an initial entity, then rely on text retrieval," which tells you little about whether an agent can combine "object appearance, local details, or spatial relations with external information."

Each task ships with a reference answer, traceable evidence and scoring rules. Alongside Accuracy for full task success, the paper reports a Process Score for the correctness and completeness of intermediate results. The authors' own read of the numbers: "These results highlight the challenge of linking visual evidence with search results to produce complete answers."

The abstract names no per-model scores and no vendors; any leaderboard sits inside the 41-page paper, not the summary.