NVIDIA Opens NemotronLabs VoiceChat 11B With Tool Calling

A model card that went up on Hugging Face today is worth a closer look, because it is the first open-weight full-duplex voice model to ship with live function calling. NVIDIA's NemotronLabs VoiceChat 11B is an end-to-end speech model that skips the usual ASR to LLM to TTS pipeline and does the whole loop in one hybrid Mamba/Transformer stack.

The build is not exotic. NVIDIA takes its Nemotron Nano v2 9B language model, puts a Fast Conformer speech encoder in front, an NVIDIA TTS decoder behind, and adds a separate output channel that emits tool-call scripts while the audio conversation keeps going. The model card reports around 450 ms latency on smooth turn taking, 480 ms on user interruption, and a #2 ranking among open full-duplex models on VoiceBench. Tool calling scores 56.1 percent average on BFCL-v3, with 82.5 percent on tool selection but only 44.2 percent on argument accuracy and 33 percent Pass@1.

Why this matters if you are not building voice agents yourself: the closest thing to this in production has been closed, hosted, and priced per minute. Having a reference full-duplex model with live tool calling that runs on A100, H100, H200, B100, B200 or RTX-6000 hardware via vLLM gives researchers and infrastructure teams something concrete to poke at, benchmark against, and copy architectural ideas from. The card explicitly positions it against PersonaPlex, Moshi and Freeze-Omni, none of which the card credits with tool calling.

The honest caveat is the license and the numbers. NVIDIA is shipping under its OpenMDW v1.1 license for research purposes only, so a startup can study the checkpoint but cannot ship a product on it. And the tool-calling scores, especially 33 percent Pass@1 and 42.5 percent on parallel tool calls, suggest this is a first-generation open attempt rather than a drop-in replacement for a hosted voice API. What the reporting does not give you is a real-world deployment cost, a third-party latency reproduction, or a comparison against closed hosted voice systems in blind tests.

The forward-looking read is that the interesting axis of competition in voice agents is moving from raw speech quality toward the mundane plumbing that makes agents useful, tool calls that fire mid-sentence without breaking the conversation. Once a reference open implementation of that plumbing exists, the closed voice APIs stop being the only place to see it working.