Found first: a primary source the press has not covered yet.
ReactHuman is a new benchmark testing multimodal LLMs on real-time response to sudden physical hazards, and it finds seven frontier models mishandle roughly one hazard in three. The benchmark requires models to commit to executable actions, not answer questions about video clips. None of the observed failures shrink with model scale.
What the source says
The paper, submitted September 9, 2026, comes from Yizhan Li, Jianxin You, Mengyang Xiong, Yinhuan Chen, Zicheng Zhao, Dekun Wu, Dongqing Zhang, and Bang Liu. The benchmark covers 17 event families, including hazards such as catching a slipping plate and dodging a falling knife, across more than 1,000 bit-for-bit reproducible scenes generated by a 240 Hz rigid-body simulator that provides annotation-free ground truth. A five-metric suite scores each model response on whether it is reasonable, safe, and physically grounded, and every committed plan is physically executed. Seven representative MLLMs were evaluated; all failed at roughly the one-in-three rate. The dataset includes adversarial objects with mismatched appearance and physics, such as a foam anvil and a steel apple, designed to probe whether models read motion or merely appearance.
Why it matters
Household robot deployment decisions are being made now, and this is the first benchmark to require physics-reactive decision-making rather than passive scene understanding. The failure pattern is consistent across all seven models: they act from trained dispositions rather than what the current scene shows, and they prioritize visual appearance over motion, missing interception geometry at meter scale even when the chosen action class is correct. Because the failures are reproducible and show no correlation with model size, they give teams evaluating embodied AI safety a concrete, measurable signal rather than a qualitative concern. Scaling alone will not close this gap.