KAIST's AFFECT Lifts DROID Policy-Evaluation Correlation to 0.79
TL;DR
- AFFECT raises the Pearson correlation between DROID world-model and real-robot policy success from 0.47 to 0.79 on the DreamDojo backbone.
- On UR5e pick-and-place, planning with an AFFECT-trained world model lifts task success from 60.0% to 68.3%.
- The objective trains generated video to respond to action perturbations as recorded video does, without repeat executions from the same state.
The problem with video world models for robotics is that they look right but react wrong. Nearly identical actions ought to produce qualitatively different futures, like a slightly off grasp dropping the cup, and current models paper over that difference while still generating convincing pixels. That gap is what a new paper on Hugging Face from KAIST's Yonghoon Dong and Jinwoo Shin tries to close.
Their objective, called AFFECT, injects temporally correlated noise into actions from a policy trained on human teleoperation, then trains the world model so that its generated video responds to that noise the way recorded video does. "We then train the world model so that its generated videos respond to this noise as the recorded videos do, without requiring repeated executions from the same initial state," the authors write.
The reported numbers: on DROID, a real Franka Research 3 setup, the Pearson correlation between policy success rates measured in the world model versus on the real robot climbs from 0.47 to 0.79 using the DreamDojo backbone, "alongside up to 23% lower FVD." On UR5e pick-and-place planning, task success rises from 60.0% to 68.3%. Across ALOHA-2 and OGBench simulation the direction is similar, though the size of the lift varies by backbone.
The results sit on top of two specific backbones, DreamDojo and Cosmos3, trained with 5-10 hours of perturbation play data per embodiment, and no outside replication yet. The paper lands in a visible stretch of robotics work on our radar, alongside coverage like NUS HINT-WAM on LIBERO-90, as sim-to-real alignment keeps getting pushed.
Originally reported by huggingface.co
Read the original article →Original headline: AFFECT Trains World Models on Action Perturbations, Lifts Policy-Evaluation Correlation 68.8% on DROID