HSS Benchmark: Top MLLM Hits 53.6% as Humans Reach 93.1%

GPT-6-astra, the strongest multimodal model its creators benchmarked, reaches 53.6% accuracy on tasks a human sample scores 93.1% on. The gap sits in a corner of visual reasoning the authors argue has gone unmeasured: the quick, implicit judgments people make at a glance. The paper, posted to arXiv as "Humanity's Sixth Sense: Benchmarking Intuitive Visual Reasoning in Multimodal Models," calls that capability a form of "humanity's sixth sense."

The authors describe the inference they are after plainly. "A quick peek determines if a vehicle can fit between two parked cars; a few seconds of video reveals who holds authority in a room," the paper writes.

HSS contains 522 open-ended tasks across 288 images and 234 video clips, totaling 17.6 hours of video. Each is grouped under a structured taxonomy of implicit temporal, spatial, social, and abstract patterns people infer at a glance.

Giving the models a workaround does not rescue them. The paper reports that an agentic setup applying "dynamic visual manipulation" to HSS "narrows but does not close the gap." The authors frame their contribution as reopening a question benchmarks had closed: HSS "establishes intuitive visual reasoning as a measurable axis and directs attention to a capability that scaling on current benchmarks has so far left behind."