SimWAM Hits 91.5 PDMS on NAVSIM Without Inference-Time Video
TL;DR
- SimWAM reports 91.5 PDMS on NAVSIM by using video generation only as a training signal, then discarding the video branch at inference.
- A pretrained video expert and lightweight action expert are co-trained via joint flow matching, with an isolated attention mask separating action from future frames.
- The authors add reinforcement learning for a compositional driving reward and report zero-shot transfer to nuScenes at substantially lower latency than prior WAMs.
For a while now, world-action models for end-to-end driving have carried an awkward tax at inference: to plan a trajectory, they generate a future video, then read the action off it. A new arXiv preprint from Zongchuang Zhao and colleagues, titled "SimWAM: A Simple World Action Model for End-to-End Autonomous Driving," proposes skipping that generation step at deployment entirely.
The design is architectural. During training, a pretrained video expert and a lightweight action expert are co-trained with what the authors call "joint flow matching," and an "isolated attention mask" keeps action prediction independent of the generated future frames. Because the two experts share no parameters and only interact through a unified attention interface, the video branch can be dropped once training finishes, leaving what the paper calls "a self-contained planner that directly predicts trajectories." The video prior contributes signal, but never latency.
The headline result is 91.5 PDMS on NAVSIM, which the authors say "surpasses state-of-the-art WAM-based planners with substantially lower latency," and they add that the model "transfers zero-shot to nuScenes." Reinforcement learning is layered on top to optimize a compositional driving reward beyond trajectory imitation.
Several things the abstract does not quantify will matter for anyone weighing the approach. There is no wall-clock latency number, no specific baseline named for the lower-latency comparison, no breakdown of what makes the RL reward "compositional," and no nuScenes score to accompany the zero-shot claim. It is an arXiv preprint, and the reported gains are on a benchmark.
Still, the design gesture is the interesting part. If a video prior really can be a training-time scaffold that gets discarded, then teams building AV stacks on WAM ideas could get the capability gains without paying the inference bill, and the video backbone becomes a swappable component that improves as open video-generation models improve. That reshapes what deployment of world-model-based planning could actually cost.
Originally reported by paper
Read the original article →Original headline: SimWAM Tops NAVSIM at 91.5 PDMS by Discarding Video Generation After Training