VIABench Tests MLLMs on Blind-Assistance Egocentric Video

A new benchmark from Nanjing University and Shanghai AI Laboratory, published on Hugging Face, tries to answer a question the general video-QA leaderboards duck: how well do today's multimodal models actually help a blind person get through their day. The answer, on the numbers the authors report, is not very well yet.

VIABench collects 761 first-person videos totaling 46.9 hours, with 14,526 annotations, and 94% of the source footage recorded by visually impaired individuals themselves rather than sighted stand-ins. The team splits the work into three tasks that map to real assistance: proactive reminders (the model has to volunteer a warning without being asked), visual question answering, and multi-turn vision-guided interaction. They also introduce Token-Level Prompt Activation Decoding, a way to score proactive reminders on offline MLLMs in a single forward pass, which they clock at roughly 4.5x faster than repeated prompting (76.85 seconds versus 344.64 on their example).

The headline result is that even GPT-5, the strongest system tested, averages 28.8 across the benchmark. Stage-one detection recall for the proactive reminder task sits under 45% for most models. The weakest sub-task across the board is Direction Deviation, which needs actual trajectory reasoning rather than single-frame recognition. Open-source models, per the paper, both hallucinate more and, tellingly, fail to internalize that the user is blind. In one qualitative example a user asks about washing-machine remaining time and InternVL3.5-8B replies that the time is 'visible on the control panel', while GPT-5 and Gemini-2.5 Pro read the value or guide the camera.

The honest caveats are worth flagging. This is a benchmark paper, not a field study, so the numbers say how models score on curated clips rather than how a person navigating a real crosswalk fares. The reporting also does not spell out consent or compensation arrangements for the visually impaired creators whose footage was sourced from YouTube, Bilibili and DouYin, and it does not translate its latency figures on an NVIDIA L20 into anything close to a phone or wearable budget.

The useful thing here is not the leaderboard, it is that accessibility teams inside the frontier labs and inside apps like Be My Eyes or Envision now have an external yardstick that specifically penalizes the failure modes that matter for this user group. That is the kind of pressure that tends to move numbers.