HIT, NUS SkeLAM Cuts End-Effector ADE 19.5% on DROID

HIT (Shenzhen), NUS's Show Lab and Ping An Tech report that SkeLAM, a skeleton-guided latent action model, reduces end-effector ADE by 19.5% and LPIPS by 11.1% on DROID compared with DreamDojo, according to the paper posted to Hugging Face on October 23.

The problem the authors flag is that latent action models learn robot dynamics from video by encoding visual transitions as proxy actions, but reconstruction objectives "encode robot motion together with object responses and other scene changes," muddying the mapping back to robot controls. SkeLAM's answer is to split those latent actions into a "kinematic component grounded by robot skeleton supervision and a residual component retaining complementary visual dynamics," then feed both back into the world model through separate conditioning pathways with branch dropout so inference can run without the future-derived residual.

A cross-modal swap trick does extra work: during training, the RGB-derived kinematic latent is replaced with its skeleton-derived counterpart while the RGB residual stays put, which the paper argues pushes correspondence "beyond alignment in latent space." A lightweight adapter then maps robot actions into that kinematic space, and the world model is fine-tuned on the predicted conditions.

Beyond the DROID numbers, the authors report motion probing that supports kinematic specialization, "zero-shot human-to-robot motion transfer in generated rollouts," and "agreement between predicted success rates and those measured in real robot executions" in open-loop policy evaluation. The abstract names only DreamDojo as a baseline and gives no per-task success numbers. It lands in a crowded week of robot world-model papers in our robotics tracker, which has logged 169 stories in the last 90 days.