MOSS-VL-Realtime: 11B open VLM watches video while it speaks

The MOSS-VL team has released an 11.3-billion-parameter vision-language model built so it can keep watching a video feed while generating a response.

"We present MOSS-VL, an open vision-language model family that treats real-time interaction -- perceiving while it speaks -- as a first-class capability," the paper states. The trick is a gated cross-attention path: the language decoder attends to vision only through that side channel, so incoming frames do not have to squeeze into the token sequence the model is currently emitting.

The headline benchmark: 66.0 versus 37.5 for the best baseline on OmniMMI Proactive Alerting. Across four streaming benchmarks, the paper reports MOSS-VL-Realtime posts "the best average on three (second on the fourth) among open-source streaming models," sweeping the three subsets that "squarely test proactive behavior."

Against Qwen3-VL-8B on the same backbone, its time-to-first-token advantage widens "from 2.8x to 5.1x as visual context grows."

The team is shipping the weights on Hugging Face under an Apache-2.0 license, per an announcement on X that also lists a 256K-token context window and Chinese and English multimodal understanding. Three checkpoints ship: a Realtime variant for streams, an Instruct variant for offline tasks, and a Base variant for continued pre-training. The abstract publishes no hardware details or per-benchmark scores beyond the OmniMMI figure.