RoboValue tests robotic value models on 15 sim and 20 real tasks

The paper, posted to arXiv on Oct 19, probes a piece of robotics research that usually sits in the background: the value model, the component that watches a policy run and scores whether it is making progress. RoboValue, from a 19-author team led by Shengbang Liu, puts those models on a shared bench across 15 simulation tasks and 20 real-world manipulation tasks, scored in zero- and one-shot settings along four axes: task-state understanding, temporal progress monitoring, failure and recovery reasoning, and value consistency.

The abstract reports errors "in fine-grained instruction grounding, history-dependent progress tracking, and recovery assessment, even in models with strong outcome and forward-progress scores," and finds that "the same semantic subtask can also receive substantially different value gains across valid execution orders."

The dataset ships in LeRobot v3.0 format with expert demonstrations and annotated trajectories, so researchers can benchmark value models independently of the policies they usually sit next to. It arrives in the same week Tsinghua opened its HandsOn benchmark, one of 167 robotics stories we've tracked over the last 90 days.