Alibaba DAMO's RynnVLA-Latent Pretrains on 30K Hours of Mixed Video, Then Adapts to Any Robot Via 3D-Grounded Latent Actions

RynnVLA-Latent from DAMO Academy, CUHK and Hupan Lab factorizes robot transitions into motion, action and camera latents, then uses a Mixture-of-Transformers policy to predict latent-action chunks before adapting to embodiment-specific controls. Pretrained on 30.1K hours (3.06B frames, 16.63M episodes) spanning human, robot and simulation video, the 4B model hits 98.8% on LIBERO, 84.7% on LIBERO-Plus and 51.9% on VLABench, and tops baselines on dual-arm real-world tasks.