Temporal Context Routing cuts script-video timing error 96%

A new preprint drops script-to-video shot timing error from 1.11 seconds to 0.042 seconds, a 96% reduction, on a 200-script benchmark.

The paper, "The Missing Temporal Link" from a team of ten authors led by Yichen Liu, spells out what current joint audio-video generators get wrong. "Current joint generators align video and audio representations on a shared temporal axis, yet the precise timing of shots and dialogue specified in a structured prompt is encoded only in the prompt's text representation and remains unaligned with the temporal coordinates of either modality," the authors write. Video and audio stay in sync with each other, in other words, while both drift away from the script.

Their fix, Temporal Context Routing, "maps the script timing onto the shared temporal axis of video and audio generation and routes each prompt's guidance to the corresponding positions in both modalities." Dialogue accuracy within a half-second window climbed from 28.3% to 84.1%.

A user study reports participants preferred TCR "on all five evaluated dimensions," and the authors say visual quality and audio-video synchronization stayed comparable to the baselines. The baseline generator itself is not identified in the abstract, and no code release is announced.