Meta and Oxford's OverSeer Anchors Multi-Agent World Models

A Meta AI and University of Oxford paper on arXiv introduces OverSeer, a multi-agent video world model that generates each agent's egocentric view alongside a set of fixed-pose auxiliary camera streams. The auxiliary views are used as a world-centric proxy so that what one agent sees stays consistent with what the other agents, and the scene itself, are doing.

The core trick for making this affordable is a specialised δ-VAE. The paper describes it as retaining the first frame's latents as an anchor and then encoding subsequent frames as residuals, cutting tokens per frame from 864 to 108 — an 8x reduction — and dropping the total compute overhead by 62.5%. With that compression, six auxiliary views cost only 37.5% of the token budget of two full agent streams.

The consistency gains are large against very weak baselines. On the authors' own Minecraft MAC-Bench, OverSeer reaches "60.2% VLM accuracy" on the Reunite at Start task "(vs. 0% for baselines)", 37.9% on Reunite Far against 0 to 6.2%, and 40.5% on Dynamic Event against 8.3 to 12.5%. Doom tasks land at 62.0 to 67.0% accuracy on the reunite setups.

One finding reads as a quiet rebuke to the field's default metric. The paper reports that "standard frame-level metrics (FID) poorly reflect structural consistency — one baseline achieved near-identical FID to OverSeer while failing 100% of consistency tasks." The multi-agent generation stack we have been scoring, in other words, has been failing a test nobody was running. This is the third multimodal world-model result on our radar this week, after ETH Zürich's VPM and Slot-JEPA.