KAIST's RoboTryOn Lifts Novel Robot Task Success 2% to 60%

A policy that could handle novel manipulation tasks 2% of the time handles them 60% of the time after training on videos of humans pretending to be the robot.

That is the headline number in RoboTryOn, a method out of KAIST and RLWRLD that treats the human-to-robot embodiment gap as a clothing-swap. "Try-on keeps the person and swaps the garment, RoboTryOn keeps the demonstrated motion and swaps the human for the robot," the abstract writes. An egocentric human video is translated keyframe by keyframe into a video of the target robot, motion is conditioned on the human hand skeleton, and an inverse-dynamics model with depth features reads off joint actions for policy learning.

The 2%-to-60% figure is an average across novel tasks on humanoid embodiments, "without any real robot demonstration of those tasks," the authors write. They also claim synthetic-only training "matches the performance of policies trained solely on teleoperated demonstrations at approximately a third of the collection cost."

The abstract is quiet on compute: readers have to go to the paper for the per-demo GPU budget, and the public page does not spell out which novel-task categories still fail. It lands in a busy week for robot-foundation-model work; our Robotics tracker logged a different video-model-to-robot pipeline, DreamX's universal latent-action learner, the same day.