Heartbeat Thinking cuts voice-agent latency 36.4% on FDBench v3

A new preprint proposes keeping a reasoning model running, and dispatching tool calls, while the user is still talking. On one benchmark, it trims mean final-response latency by 36.4%.

Posted to arXiv, the paper reports that on FDBench v3, speculative tool coverage rose from 34.67% to 50.67% and mean final-response latency fell from 6.90 to 4.39 seconds against a SHANKS-style windowed baseline. "Tool-using voice agents often leave users waiting after speech ends, even when actionable information arrives earlier in the request," write Zhenyu Zhang and co-authors. Their diagnosis of the usual failure mode is flat: speculative approaches that start work during speech "can truncate a tool call before its arguments are complete."

Their fix, called Heartbeat Thinking, periodically re-invokes a Thinker on the latest partial transcript while letting earlier invocations and dispatched tools finish. A separate Responder takes over at detected end-of-speech, checking completed preparation against the full request and reusing what still applies. The abstract frames the goal as turning "listening time into reusable preparation."

The authors evaluate across SpeechFCEval, FDBench v3, and AudioCRAG, and report "benchmark-dependent effects on final task quality." The abstract publishes only the FDBench v3 coverage and latency figures; per-benchmark numbers for the other two benchmarks are not included.