NVIDIA Releases Audex-30B Unified Audio-Text LLM on Hugging Face

Tucked into this week's arxiv drops is a release worth pausing on: NVIDIA has shipped a 30 billion parameter audio-text model called Audex, built on top of its Nemotron-Cascade-2-30B-A3B MoE backbone, and put the weights on Hugging Face. The paper argues it handles speech recognition, translation, text-to-speech, audio generation and speech-to-speech in one model while preserving the text backbone's reasoning with 'marginal or no regression.'

The architecture is the interesting part. Rather than bolting a speech encoder onto a language model as an adapter, Audex uses 'a single Transformer decoder' where audio inputs are 'encoded and projected into the text embedding space,' and 'quantized audio output tokens are treated uniformly' with text tokens during generation. In practical terms, the same inference stack that runs a text LLM runs this one. Training used a heavy audio corpus, 157.4 billion audio tokens alongside 320.5 billion text tokens, followed by 'text-only Cascade RL and multi-domain on-policy distillation.'

Why that matters if you are not training models yourself: today's production speech stacks are usually stitched together, a hosted transcription service, a separate TTS vendor, a third service for audio generation. A single open-weights model that covers all of those while keeping the underlying text reasoning is a different shape of thing to build on, especially for teams that want the audio stack on their own hardware.

The honest caveats. The abstract is where the specificity lives on arxiv right now, and it claims 'state-of-the-art' across the audio tasks and 'marginal or no regression' on text without publishing the head-to-head benchmark tables in the summary I could retrieve. What the reporting does not give you yet is direct numbers against the leading closed audio systems, real-world latency and cost on typical serving hardware, or the coverage picture across languages and accents. It also does not address the obvious open-weights safety questions around voice cloning.

For NVIDIA, this is another push to make the open ecosystem good enough that closed audio APIs have to defend their pricing on quality alone. For everyone else, the question worth watching is whether an open unified model gets close enough to the hosted specialists that self-hosting, fine-tuning and one bill instead of three become the easier choice.