SJTU, Shanghai AI Lab Open FilmFlow for 3D Camera Shots

Shanghai Jiao Tong University, Shanghai AI Laboratory and UCLA have released FilmFlow, an agentic system that converts a text instruction or a reference clip into an executable camera trajectory inside a 3D Blender scene. A Director agent composes shots from 22 atomic camera operations — translation, rotation and framing primitives with overlapping time intervals — while an Interpreter extracts cinematographic intent, a Scout picks the opening viewpoint, and a Critic rates each render and feeds revisions back into the plan.

The companion benchmark, CinemaBench, is the part most useful to outside teams. It bundles 150 Blender scenes (73 dynamic, 77 static), five camera instructions per scene for 750 text-directed tasks, and 60 reference videos for video-to-video evaluation. Fifty scenes seed an in-context-learning example bank; the remaining 100 are held out for the reported numbers — 500 T2V and 40 V2V test tasks.

The authors score everything with external evaluators that share no weights with FilmFlow: VideoReward for text alignment and motion quality, the DOVER composite for overall quality, and a CameraBench-trained Probe for camera-motion matching. In a blinded user study, forty-two participants were each assigned 10 T2V and 10 V2V trials, producing 420 T2V and 397 V2V completed rankings. FilmFlow was ranked first in "61.0% of T2V judgments and 63.2% of V2V judgments." The ablations are tighter than the headline suggests: on V2V Probe, the no-ICL variant actually edges the full system, "0.766 vs. 0.750," which the paper attributes to Probe's auxiliary role.

The baselines are a who's-who of recent camera-generation work — LAMP, GenDoP and CinemaTraj on the text side, LingBot-Map, VGGT-Ω and a zero-shot VLM on the reference-video side. All four FilmFlow agents, and the VLM baseline, run on "gpt-6-astra with medium reasoning effort," so the comparison is one orchestration recipe against another on the same backbone.

The paper lands in a crowded week for agent work — our tracker logged 525 agent stories in the last 90 days, including a CMU-Stanford agent LEGO-building result also posted today. Future work, the authors write, will investigate "whether smaller models can achieve comparable performance through task-specific fine-tuning" and extend the system to jointly automate scene construction and camera planning.