Alibaba DAMO's RynnValue swaps preference labels for time

A new paper out of Alibaba's DAMO Academy goes after what most robot-learning groups now consider the real ceiling on scaling reinforcement learning: not policy capacity, but reward supervision. RynnValue, from DAMO with Hupan Lab, replaces the usual preference labels and normalized progress signals with something you can read straight off a clock — temporal distance, the directed 'cost-to-go' from an observation to a language-specified goal.

The practical consequence is that supervision comes free with the data. The authors report training on over 7,000 hours and roughly 3M instruction-conditioned clips spanning real-world, simulated, and egocentric trajectories, without preference pairs or progress annotations. Their 8B model, RynnValue-8B, reaches an average Kendall's tau_a of 0.675 on the RBM-EVAL-OOD benchmark, ahead of the fully preference-supervised state of the art at 0.655 and more than double a progress-only counterpart at 0.292.

The bigger claim is downstream. When the temporal-distance signal is converted into a dense reward through potential-based shaping, the paper says real-world policy success on top of the strongest baseline goes from 52.5% to 72.5% online, and from 63.8% to 82.5% offline. If that survives outside evaluation, it is one of the cleaner arguments to date that 'value foundation model' deserves a spot next to the language and visuomotor foundation models coming out of the same labs, and it fits a run of Chinese open-source robotics work we've been tracking in our robotics coverage.

A few things to be careful with. The 52.5% to 72.5% online figure sits behind a single baseline and a team-run evaluation, and the abstract does not break it down by task. The authors also acknowledge that raw episode boundaries do not correspond to semantic completion, so their pipeline uses 'subtask segmentation and cutoff relabeling' to define completion points, which is quieter labor than 'labels come from timestamps' first suggests.

If temporal distance really does scale with heterogeneous embodied logs the way the authors argue, a lot of previously awkward data — sim rollouts, egocentric human video, mismatched embodiments — becomes reward-training fuel, which is exactly the corner where paid annotation has been slowing everyone down. Recent reports that Surge AI and Mercor are selling training data to Chinese labs are one sign of how tight that supply has gotten.