NVIDIA BirdDreams Jointly Generates Driving Video and BEV

In 375 pairwise judgments by 25 participants, surrounding-vehicle behavior from NVIDIA's new driving world model, BirdDreams, was preferred 57% of the time versus 24% for a perspective-only baseline from the same lab. The remaining 19% were ties. The paper, posted Nov 13, describes an autoregressive model that couples two video diffusion transformers through joint attention. One produces first-person driving video, the other a schematic bird's-eye view at a fixed metric scale. From a single camera image it generates both streams, with no HD map or vehicle-state inputs required.

The authors' pitch for the design: the BEV 'provides an explicit training target for vehicle motion relative to the road,' letting a pixel displacement mean the same metric distance anywhere in the view. In the perspective-only baseline, by contrast, 'vehicles can disappear or make implausible maneuvers at complex intersections.' Across PhysicalAI-AV and nuScenes, BirdDreams reduces minimum average displacement error (minADE) by 21% on PhysicalAI-AV and 37% on nuScenes relative to that OmniDreams baseline. With a ground-truth ego trajectory supplied, the paper reports 'reducing ADE by 35% on PhysicalAI-AV and 36% on nuScenes relative to OmniDreams (Plücker).'

The model is a roughly 4-billion-parameter dual-branch Mixture-of-Transformers, trained from an OmniDreams checkpoint on about 670 hours of driving from RDS-HQ-1M, then post-trained on the nuScenes train split for the public comparison. Training ran on 64 H100 GPUs with a batch size of 64. Each BEV frame covers a 69.3-meter square so that pixels map consistently to meters. A footnote states the work was done while the authors were at NVIDIA.

Two caveats the paper surfaces itself. The automatic trajectory metrics reconstruct ego motion from the generated perspective video using Depth Anything 3, not direct supervision, so the comparison sits on top of a separate estimator. The human study is 375 pairwise judgments from 25 participants, drawn from a pool of 119 video pairs. For per-frame Image Quality, BirdDreams ranks first on PhysicalAI-AV but second on nuScenes behind DrivingDojo, so the gains concentrate in trajectory realism rather than pixel fidelity.

NVIDIA has been the subject of 261 of our tracked stories in the last 90 days, most of them about export controls and chip supply. This one lands on the AV-simulation side, where Applied Intuition is reportedly raising $1.5B around the same closed-loop problem.