Hugging Face Paper: Robot Policies Don't Need Video Generation

Robot policies can match or exceed their video-generating counterparts without generating a single pixel at train time, according to a new Hugging Face paper titled "Vision-Language-Action or World-Action? Just Action Models." The authors call the recipe JAM: a pretrained visual backbone, an action expert, and a "foresight loss" that trains current-observation features to predict frozen encodings of future frames through a prediction head thrown away after training.

The headline numbers come from LIBERO. "JAM improves the success rates from 97.0%/55.3%/91.2% to 98.2%/64.5%/91.4%, respectively," the paper reports, comparing matched FastWAM baselines on LIBERO, LIBERO-Plus and RoboTwin. On the perturbed LIBERO-Plus suite (camera, lighting, language and layout changes), the gain against ImageWAM is larger, from 65.27% to 79.95%. The compute line is where the argument sharpens: with cached foresight targets, "JAM reduces estimated online training FLOPs per sample per update by 73% relative to FastWAM."

The authors frame the debate directly in the abstract: "The vision-language-action (VLA) and world-action model (WAM) communities debate whether control policies should inherit pretrained knowledge from vision-language models or from visual generators." Their claim is that the gap "cannot simply be attributed to any one of them": it bundles backbone initialization, future-observation objective, and whether the generation machinery stays at inference. Decouple those and the generative branch is doing less work than the field assumed.

The cross-backbone table is the other interesting beat. On LIBERO-Plus, a vision-language backbone (Qwen3.5-2B) reaches 81.14%, the generative Ovis-U1 reaches 79.95%, and a self-supervised DINOv2-L lands at 70.39%, all under the same foresight recipe. "strong foresight performance is attainable with either generator-initialized or vision-language backbones," the paper concludes. It joins two other robotics results landing on our tracker today, NVIDIA and KAIST's ThinkWorld VLA and the Rollback world-model paper.

Training ran on 8 NVIDIA GH200 GPUs with public code built on FastWAM and ImageWAM. The abstract does not publish per-suite breakdowns, and the measured GPU-time reduction on the Wan variant (67.3%) is lower than the FLOP figure.