StereoBind binds stereo audio to on-screen source motion

StereoBind, a new framework from Hanmo Chen and nine coauthors, pins generated stereo audio to the motion of its on-screen visual source. The authors give the property a name: Dynamic Spatial Correspondence, defined as "the perceived sound location to evolve consistently with the motion of its corresponding visual source."

Described in a preprint posted to arXiv on September 30 and revised October 8, the method routes motion tracks through three components. Visual Motion Binding "establishes source-aware audiovisual correspondence." A Spatial Track Encoder "captures absolute source positions." Residual Track RoPE "models relative motion."

To train and score the system, the team built StereoWorld-29K, a stereo audio-video dataset with paired motion tracks, and StereoWorldBench, an evaluation benchmark for audiovisual spatial consistency. The abstract reports that StereoBind "substantially improves spatial alignment in stereo audio generation over existing models while preserving overall audiovisual quality." No per-metric numbers appear in the abstract itself.