paper web signal

Direct3D-V3 Discards the VAE for Raw-Voxel 3D Diffusion

TL;DR

  • Direct3D-V3 drops the variational autoencoder and runs its diffusion transformer directly on raw SDF voxel grids during both training and sampling.
  • The project page reports the method pushes to up to 1536³ voxel resolution, supervised by rendered normal losses after Marching Cubes conversion.
  • The paper appears in ACM Transactions on Graphics Vol. 45 No. 6 at SIGGRAPH Asia 2026, with code on GitHub and a pretrained model on Hugging Face.

"Native 3D generation has largely been built around latent diffusion," the authors write in the arxiv abstract, and the paper's point is to step away from that premise. Direct3D-V3 drops the variational autoencoder that triplane, vecset and sparse-voxel pipelines all use to compress geometry before modeling it, running its diffusion transformer directly on raw SDF voxel grids during both training and sampling.

The project page describes the result as "High-resolution 3D generation directly on signed distance fields, at up to 1536³ voxel resolution."

Two technical patches keep the raw-voxel objective from falling apart. The authors "introduce an implicit manifold regularization over neighboring voxels to suppress non-watertight surfaces," and convert predicted SDF grids into meshes with Marching Cubes so a rendered normal loss can supervise the network against its own output. A separate auxiliary head predicts the conditioning image's camera parameters, nudging the model toward image-geometry alignment.

The paper runs in ACM Transactions on Graphics, Vol. 45, No. 6, Article 244, at SIGGRAPH Asia 2026. Code sits at github.com/DreamTechAI/Direct3D-V3, and a pretrained model is on Hugging Face.

The abstract publishes no head-to-head numbers against the VAE pipelines it positions itself against.