Tencent ARC's GAE halves 3D camera error in world generation

A geometry-native autoencoder from TencentARC cuts camera-trajectory error in half on the RealEstate10K benchmark when it replaces the standard latent inside a video generator, according to a paper posted on arXiv. The same swap lowers FVD, a video-quality metric, by 12.7% on RealEstate10K and 23.1% on DL3DV.

The authors frame 3D-consistent generation as a representation problem rather than a modeling one. "Generators typically evolve appearance-centric latents, while perception models recover geometry in a semantically rich space that encodes cross-view structure," they write. Their fix, called GAE, "reparameterize[s] a geometry foundation model's features into a compact latent space for generation," with a single latent "jointly decodable to appearance, depth, cameras, and point maps."

The comparisons hold the generator and training protocol fixed and change only the latent, which is how the paper isolates GAE as the source of the gains. Beyond the two FVD numbers and the halved trajectory error on RealEstate10K, the abstract does not publish per-scene breakdowns, comparisons to explicit 3D-aware baselines, or which geometry foundation model supplies the features.