Falcon Cuts Audio-Visual Speaker-Isolation Compute 20.2x for Live Video Calls

Found first: a primary source the press has not covered yet.

A paper by Wendi Sang, Kai Li, Yifan Li, Jianqiang Huang, and Xiaolin Hu introduces Falcon, a causal audio-visual target speaker extraction model that reduces computation by 20.2 times and GPU inference latency by 11.1 times against the causal AV-TFGridNet baseline, while cutting parameter count by 91.9%. The authors simultaneously argue that every existing AVTSE benchmark is too easy and introduce RealSSA to replace them. The paper was submitted October 23, 2026.

What the source says

Falcon performs multi-scale time-frequency separation in a single encoder-decoder pass rather than the heavier architectures used by prior causal methods. To suppress near-field speaker leakage, the authors add an encoder-space speaker contrastive loss, which they report transfers across backbones at zero inference cost. The paper introduces RealSSA in two parts: RealSSA-Sim generates scenes with 1 to 5 dynamic near-field interfering speakers and 10 to 15 far-field speakers, while RealSSA-Real is a held-out real-recorded evaluation set. Falcon achieves state-of-the-art extraction quality on LRS2-2Mix, LRS3-2Mix, VoxCeleb2-2Mix, and both RealSSA splits.

Why it matters

Real-time speaker extraction requires causal, low-latency models, and the best-performing ones today carry compute costs that make edge or large-scale deployment difficult. A 20.2x compute reduction and 11.1x GPU latency cut, if confirmed under peer review, change what hardware can run a competitive model. The RealSSA benchmark may be the more consequential contribution: standard evaluation sets use two-speaker mixtures, which do not reflect the dynamic multi-speaker near- and far-field conditions of actual video calls. A benchmark that is harder to game changes which models the field counts as good.