SolarRenderer renders real-time 768p video from scene structure

SolarRenderer, a paper posted to Hugging Face by researchers at NUS, Lightspeed, CUHK(SZ), SLAI and HKUST(GZ), adapts a frozen MiniMax-H3 33B video backbone with two lightweight LoRAs so that coarse scene structure — block-outs, depth maps, pose skeletons, point clouds, pencil drawings — can be turned into finished video in four denoising steps.

The pitch, in the authors' own words: structure-guided distribution matching "renders in four steps, 12.3 times faster than 50-step sampling, while preserving the learned control." A separate variant extends the design to streaming and "renders 768p video in real time."

The architectural trick is that structure is passed in without a dedicated head. "Structural videos enter the native reference pathway of the backbone at the positions of the frames they describe, without a type identifier and described only in plain text, so that a single control LoRA follows arbitrary structural representations," the abstract says. That is what the paper credits for the generalisation claim: the model "generalizes to 11 unseen representations, including pose skeletons, drawings, and point clouds."

Training uses "a frame-aligned corpus of 136K clips from game engines, simulators, and YouTube videos." Code, model weights and the dataset are released on the Enderfga Hugging Face and GitHub namespaces. The abstract reports state-of-the-art on an in-distribution test set and a 16-representation generalisation benchmark, but publishes no per-representation accuracy breakdown.

It lands the same day as Skywork's Matrix-Game 4.0, another open-weights push into real-time playable-world video.