Stability AI's 4Director Steers Video Models With Rigid 3D Meshes

Stability AI and University of Illinois Urbana-Champaign researchers published 4Director, a video world model that takes a per-object 3D mesh and one rigid transformation per frame as its control input, instead of image-plane tracks or blobs. Each object is "reconstructed once from the input image as a canonical mesh" and then moved frame by frame; the scene is rendered as a depth video and fed through a trainable Motion Adapter into the pretrained Wan2.1-VACE-14B generator.

The paper's framing is that existing methods control objects "only coarsely, through image-plane cues that are ambiguous in depth and rotation or through 3D tracks and blobs that lack complete geometry."

The authors' numbers come from RealCOD-Rigid, "a new dataset of 20,774 clips annotated with rigid 3D scenes by an automatic pipeline." On it, 4Director posts an Identity-Gated IoU of 60.4 against VerseCrafter's 54.8, and a rotation error of 3.65° against SymphoMotion's 3.89°. Human raters in a user study scored its control accuracy 4.82 out of 5, with the next-best system at 2.39.

That control-accuracy gap is the headline figure, and it is also the one most dependent on self-constructed evaluation: both the test set and the main object-control metric, Identity-Gated IoU, are introduced in the same paper. Baselines were MotionCtrl, Perception-as-Control, SymphoMotion and VerseCrafter. The work lands in a steady run of controllable video research we have been tracking, with 46 stories in 90 days on video generation alone.