Gen4P turns video generators into single-pass 4D perceivers
TL;DR
- Gen4P processes clean video latents in one encoder pass and predicts 4D attributes through a lightweight cross-attention decoder, skipping iterative denoising.
- A pretrained video-depth LoRA is bolted on as a frozen geometry prior, improving performance without adding extra trainable parameters.
- The authors claim state-of-the-art or competitive results on 3D point tracking, video depth and camera pose estimation; the abstract reports no numbers.
A pretrained video generator can be repurposed into a feed-forward 4D perceiver without iterative denoising. That is the central claim of Gen4P, a preprint from Yitong Xia and Chuanxia Zheng of NTU's Physical Vision Group with Zeren Jiang of Oxford's Visual Geometry Group.
The abstract is blunt about the move: "a pretrained video generator can be repurposed into a feed-forward 4D perceiver without iterative denoising." Gen4P "processes clean video latents in a single encoder pass and predicts the requested 4D attributes through a lightweight cross-attention decoder." On-demand queries handle spatial sampling, continuous-time queries handle temporal. A pretrained video-depth LoRA is plugged in as a frozen geometry prior, the authors write, "improving overall performance without adding extra trainable parameters."
The headline result they report: "state-of-the-art or competitive performance on tasks of 3D point tracking, video depth and camera pose estimation." The abstract publishes no benchmark numbers, no named baselines, no datasets. Qualitative in-the-wild demos live at gen4p.github.io.
The preprint drops into a run of monocular-video geometry work our tracker has been following this week, including Tencent Hunyuan's WorldRecon.
Originally reported by huggingface.co
Read the original article →Original headline: Oxford's Gen4P Turns Video Generators Into Single-Pass 4D Perceivers, Hits SOTA on 3D Point Tracking