SIS-Bench probes MLLM self-awareness on 4,856 UAV tasks

Autonomous drones are drifting from demos into logistics and inspection work, and a new benchmark suggests the multimodal models that would run them still have a lopsided sense of the world. In a paper posted to arXiv, Zhishan Zou and colleagues introduce SIS-Bench, a test set built from 1,646 real-world UAV videos and 4,856 question-answer pairs across 13 tasks. The stated goal is to score what current MLLMs actually understand about the space around a drone versus what the drone itself is doing inside that space.

The framing is what makes the result interesting. Rather than treating a drone's vision stack as a single 'understand the scene' problem, the authors split the evaluation along two axes, the environment and the agent, and layer a cognitive hierarchy on top: perception, then memory, then reasoning. Their headline finding is 'a clear imbalance between spatial cognition and self-awareness' in current MLLMs, with progressive performance degradation as tasks step up from perception through memory to reasoning. Models can often describe what is in front of the drone better than they can describe what the drone is doing.

The proposed fix is worth flagging. The authors explored motion-aware representations that fold agent dynamics, via optical flow, into the visual features, and report that this consistently improved perception and memory tasks. That is a hint that a lot of what current systems miss about 'self' can be recovered from motion signals the pipeline already has, rather than by adding a new sensor stack.

The honest caveat is that the material I could pull from the abstract does not name specific frontier models or publish per-model scores, so take 'MLLMs are worse at self than space' as the paper's framing, not a settled leaderboard. What the write-up also does not give you is how a failure at the reasoning level translates into a real UAV control loop under wind, GPS dropout, or occlusion, which is the operationally interesting question for anyone actually flying these systems.

Even so, the direction is useful. For teams building UAV autonomy on top of general-purpose MLLMs, SIS-Bench is a public target that pushes vendors to report on the self axis and not just on 'did the model recognise the parking lot.'