GPT-5.5 Scores 10.6% on ActiveVision, Humans Hit 96.1%

The interesting finding from a new arXiv paper isn't that a frontier vision model failed a new benchmark, that happens weekly, but the specific shape of the failure and the fact that the models cannot patch it by writing their own code.

The benchmark, called ActiveVision, contains 17 tasks across 3 categories designed, in the authors' words, to "force repeated visual perception rather than a single static description." GPT-5.5 at the highest exposed reasoning-effort tier solves 10.6% of items and scores zero on 11 of the 17 tasks. Claude Fable 5, which the authors note tops most reasoning and coding leaderboards, manages 3.5%. Three human participants averaged 96.1%.

The framing worth carrying forward is the authors' description of what these tasks are actually measuring. They describe human vision as a closed loop where "gaze is continuously redirected by intermediate hypotheses rather than a single snapshot." Most existing vision benchmarks let a model look once and answer. This one is set up so a single glance isn't enough. You have to look, form a guess, then look again somewhere else, and that is where the models collapse.

The part that stopped me was the code-execution finding. A natural response to a vision gap is to give the model tools, let it write Python, call OpenCV, crop the image, run OCR, whatever it wants. The authors report that much of the gap persists even when models write and run their own vision code, because that code is unreliable on realistic imagery and catching its failures itself requires the active perception the models lack. The tool workaround only helps if you can already see well enough to check the tool's output.

The honest caveats: this is one paper, the results have not been independently replicated, and the abstract does not name the 17 tasks or the 3 categories, so it is hard to judge how contrived the setup is. What the paper doesn't give you is evidence that a different architecture would close the gap. But for anyone building on top of hosted MLLMs for tasks that involve iterative inspection, think quality control on production lines, radiology triage, satellite imagery review, robotics, the takeaway is uncomfortable. The leaderboard leader may be the wrong tool, and the model itself cannot tell you that.