Speculative Listening trims full-duplex voice AI lag by 73%

A new preprint on arXiv claims that in full-duplex voice models, which listen and speak in fixed time chunks, the next audio chunk does not change the model's output in 95.3% of chunks. When it does change the output, 88% of the time the change begins at the very first token.

The authors, writing in "Decode Before You Hear: Speculative Listening for Chunk-Interleaved Full-Duplex Speech Language Models," turn that observation into a decoding trick. During the idle compute window while the model waits for the next audio chunk to arrive, it drafts an output assuming the chunk will be silence. For cases where changes would begin at the first token, it also drafts a second branch from the runner-up first token. All drafts are held in a "cross-chunk hypothesis pool" and verified once the real audio shows up.

The reported numbers, on an eight-task benchmark the authors call TickTock-Bench: "Speculative Listening speeds up speak chunks by 2.6-2.8x and tool-call chunks by up to 23.4x." Average decoding delay before a response starts drops 73%, to 74-241 ms, a range the paper flags as "close to the 200 ms gap between human turns." Maximum lag behind real time falls 94.4%.

The approach is pitched as a "training-free, lossless framework for input-time self-speculative decoding," meaning no fine-tuning and, per the authors, no change in output quality. The four models used go unnamed in the abstract, and TickTock-Bench is the authors' own benchmark. No external replication yet.