Sharpa-SJTU DexForesight Lifts Dexterous Policy Success to 60.5%

Sharpa Robotics and Shanghai Jiao Tong University's joint embodied-AI lab, with co-authors from CUHK and Fudan, posted DexForesight on Hugging Face Papers on November 11, reporting that a dexterous-manipulation policy can inherit action-conditioned spatial foresight from a world model during training and run without it at deployment. The paper says the recipe lifts overall success on 11 DexJoCo simulation tasks from 49.4% for its GR00T N1.7 backbone to 60.5%, with policy inference slowed by only 3.0 ms per step on the authors' real-robot setup.

DexForesight first trains an action-conditioned spatial world model on approximately 370 hours of demonstrations across 60 tasks and five categories, using scene depth estimated offline from stereo RGB with StereoAnyVideo and kinematic hand geometry derived from recorded joint states. The model predicts 3D displacements of fixed points on the hand together with visual feature changes under demonstrated controls. The authors then freeze that teacher, align the deployed policy's query tokens to the world-model queries, and strip out depth, future decoding, and the world model itself at inference. 'At deployment, only the policy is retained, operating on RGB observations, language, and robot state without depth input or explicit future prediction,' the abstract states.

On three real-robot tasks (Close Box, Insert Battery, and Screw On Lid), executed on a Sharpa North robot with 22-DoF Sharpa Wave hands, DexForesight hits 58.3% success across 20 trials per task, versus 50.0% for WLA, 48.3% for GR00T N1.7, 30.0% for Cosmos3, and 10.0% for Fast-WAM. A linear ridge probe of the deployed latents predicts 3D hand-point positions at 0.54 s with 10.01 mm error, down from 13.59 mm for the architecture-matched QwenVLA baseline, a 26.3% reduction the authors read as evidence that predictive hand-motion information survives after the teacher is removed. The paper notes the baseline comparison mixes architectures, so 'architectural and training differences preclude treating either as a one-variable causal control.'

Ablations put most of the real-robot gain on the depth-equipped teacher: removing offline depth cuts success from 58.3% to 43.3%, with Screw On Lid falling to 35.0% and Close Box to 25.0%. It is the second SJTU-affiliated manipulation result to cross our robotics tracker in two days, following SJTU-Noematrix's SWAM yesterday.