LeWAM lifts JEPA contact-rich robot success rate to 89.7%
TL;DR
- LeWAM is an end-to-end JEPA for robot control that jointly trains next-latent-state prediction, action flow matching, and an isotropic Gaussian latent constraint.
- Against prior JEPAs in visual planning, success rose from 28.6% to 89.7% on contact-rich tasks and 10.9% to 46.2% on long-horizon tasks.
- The same single model does direct policy execution and plans by evaluating sampled actions 32.3 times faster than prior JEPA pipelines.
A new end-to-end JEPA for robot control posts an 89.7% success rate on contact-rich tasks, up from 28.6% for prior JEPA visual planners, and plans 32.3 times faster. The paper, LeWAM: End-to-End World and Action Modeling with JEPAs by Minghao Fu, Tavis Siebert, Eryk Halicki and Randall Balestriero, drops the usual multi-stage JEPA pipeline, in which visual representations and action-conditioned dynamics are trained before a control policy or solver is added on top, in favor of one model trained jointly.
The design has two parts, as the abstract puts it: a "joint training objective over next latent state prediction and action flow matching," and "an isotropic Gaussian constraint on the latent state." That Gaussian constraint is what lets the same model both execute a policy directly and plan by evaluating sampled actions against its own learned dynamics.
The headline numbers: 28.6% to 89.7% on contact-rich tasks, 10.9% to 46.2% on long-horizon tasks, and "plans 32.3 times faster" than prior JEPAs in visual planning. On behavior cloning in simulation and on physical robots, the authors call LeWAM "competitive with established methods" without putting a figure on it.
The abstract names no specific benchmark suite, no baselines by name, and no wall-clock numbers behind the 32.3x speedup. The authors' interpretability section argues the learned representations are "control-relevant," with action distributions "effectively guiding test-time planning and control."
Originally reported by paper
Read the original article →Original headline: End-to-End JEPA Robot Controller: 89.7% vs 28.6%, Plans 32× Faster