π⁴ Packs 4D Geometry and Motion Into a 0.5B-Param Forward Pass
TL;DR
- π⁴ bundles dense geometry, camera poses, metric scale, 2D/3D point tracks and moving-object masks into one 0.5B-parameter forward pass from uncalibrated RGB.
- The authors report 'best overall average rank' in geometry evaluation and 'competitive performance' on 2D/3D point tracking and moving-object segmentation.
- Trained with contexts of at most 64 views, π⁴ claims to handle scenes with thousands of input images in a single forward pass.
A single 0.5-billion-parameter model that takes RGB pictures, whether a single image, an unordered collection of stills, or a video sequence, and in one forward pass returns dense geometry, camera poses, metric scale, 2D and 3D point tracks, and masks for anything that moved. That is what π⁴, posted to arXiv on 17 November by Yifan Wang, Weiguang Zhao, Ni Yang, Haoyu Guo, Junting Dong, Weinan Zhang and Chunhua Shen, claims to do.
The architecture couples a geometry branch with what the authors call a 'geometry-conditioned motion branch, which predicts image-space correspondences that are lifted into 3D using reconstructed target-view geometry.' In plain terms: solve the scene first, then let that solution tell the motion head where to look.
The size is the pitch. π⁴ comes in at 'approximately 0.5B parameters,' which the paper calls 'less than half the size of most baselines,' and on the authors' numbers takes 'best overall average rank in geometry evaluation,' with 'competitive performance' in 2D/3D point tracking and moving-object segmentation. No per-benchmark scores or baseline names appear in the abstract.
A second scaling claim: the model was trained with contexts 'of at most 64 views' but is said to produce reconstructions and tracks 'including large-scale scenes with thousands of input images.' Training-to-deployment context gaps of that magnitude normally require separate stitching or chunking.
It lands on a busy day for 4D vision work. TCP4D, a tool-tracking-plus-trajectory model aimed at robots, posted alongside it in our computer vision feed.
Originally reported by huggingface.co
Read the original article →Original headline: Meta's π⁴ Foundation Model Unifies 4D Scene Geometry, Tracking and Motion in One 0.5B-Parameter Forward Pass