Fudan's F2G-Pose Jointly Predicts 6D Pose, Size, Shape at 28 FPS

Fudan University researchers describe a single-pass neural network that estimates an object's 6D pose, metric size, and completed 3D shape from one RGB-D frame at 28 FPS, which the paper on Hugging Face reports is roughly seven times faster than the diffusion-based GenPose++ baseline it compares against.

F2G-Pose "predicts category-level pose, metric size, and camera-frame completed shape from a single RGB-D observation in one feed-forward pass," the authors write. The system "lifts dense visual foundation features to partial point clouds, encodes local geometry with DGCNN tokenization," then routes tokens through a Mixture-of-Experts Transformer with eight experts and top-2 routing. The task network is trained only on pose-annotated synthetic SOPE data; the RADIO backbone is frozen and externally pretrained.

On SOPE the paper reports 43.7 AUC at IoU50, mean rotation error 15.7°, and mean translation error 0.88 cm. On the real-world ROPE benchmark it reports 27.8 AUC at IoU50, up from 19.1 for GenPose++, while the full model including shape completion runs at 23.7 FPS. Robot manipulation trials on an ARX arm logged 74 of 80 picks and 68 of 80 placements under a 50 ms perception budget.

The paper is candid about failure modes: corrupted depth on thin or reflective surfaces, heavy occlusion hiding asymmetric parts, and symmetric low-texture objects where rotational ambiguity produces 90° or 180° errors. Code and pretrained models are promised at the project page.

The release lands in a crowded moment for CAD-free manipulation; our robotics tracker logged 170 stories over the last 90 days, including a HIT and NUS end-effector paper the day before.