Kling Paper: Video Bottleneck Shifts From Rendering to Semantics

The Kling team has put out a long technical report arguing that the rendering problem in short-form video generation is largely solved, and that the gating constraint has moved upstream. "We argue that generation quality is the projection of multimodal understanding through the entire data–conditioning–alignment–evaluation chain," the paper on Hugging Face states, summarizing observations gathered across successive generations of Kling.

The artifact carrying the argument is a family of multimodal LLMs built to sit upstream of the generator. Kling-MLLM ships in three variants: a 27B dense model, a 35B-A3B mixture-of-experts, and a 397B-A17B MoE. All are initialized from checkpoints in the Qwen3.5 through Qwen3.8 model families. The towers handle images, video, audio, synchronized audio-video and interleaved inputs, with a dual-branch Mel-VAE audio design on the larger variants.

Most of the report is caption discipline. The authors specify what they call a semantic contract made of six information families: Subject & Action, Space & Composition, Cinematography & Light, Time & Shots, Visible & Audible Text, and Aesthetics. Each comes with named failure modes if a family is omitted. Skip the cinematography fields and you get what the paper calls "Generic camera behavior, appealing but irreproducible style." Skip the subject-action detail and the generator produces "Static posing, role swaps, unresolved actions."

Reference-grounded captioning extends that contract with three conjunctive properties the authors label scoped binding, residual sufficiency and authority consistency. One supervision move the paper highlights as a data-scaling idea: any two videos with sufficient visual or acoustic similarity can be pressed into use, with one as reference and the other as target. The caption describes what they share and what changes. The videos, the paper notes, "need not originate from an editing process," and similarity can be partial.

On evaluation the authors argue aggregate scores reveal little, because a single failed clip can trace back to captions, representations, reward models or evaluators. They push for task-specific, decomposed protocols instead.

This is the second Kling-affiliated paper on our video-generation tracker this week.