Kyutai AURA Streams Audio-Driven Avatars at 20.7 fps on One GPU
TL;DR
- AURA 1.3B streams 576p audio-driven talking-head video at 20.7 fps on a single GPU; AURA 5B streams 720p at 9.7 fps.
- The model generates chunk by chunk with a rolling KV cache, removing the multi-second audio look-ahead bidirectional diffusion models require.
- Kyutai is releasing both model weights alongside a dataset of 30K synthetic videos with precomputed teacher trajectories.
Kyutai's new audio-driven video model, AURA 1.3B, generates 576p talking-head video at 20.7 fps on a single GPU. The larger AURA 5B produces 720p at 9.7 fps. Both work chunk by chunk, with only the current audio chunk required to start generating.
The authors frame the gap they are closing. 'Current audio-driven video diffusion models can synthesize realistic talking-head videos but generally require dozens of diffusion denoising steps over multi-second video chunks, making them unsuitable for streaming applications such as voice assistants and virtual avatars,' the paper reports.
The speed comes from distillation rather than a new backbone. AURA 1.3B runs '23x faster than its own bidirectional teacher and reaches a quality comparable to TalkVerse at a lower resolution', while AURA 5B 'matches its TalkVerse teacher at 720p while running 17x faster'. The recipe is two stages: first train a causal student to match teacher trajectories using a chunk-wise sliding-window causal attention mask, then apply Distribution Matching Distillation with backward simulation to close the remaining quality gap. A rolling KV cache carries state between chunks.
Kyutai is releasing both model weights plus 'a dataset of 30K synthetic videos and their precomputed teacher trajectories'. A live demo runs at aura.kyutai.org. It lands in a steady run of open-weight streaming video work we have been tracking this month.
The abstract names TalkVerse as the quality bar but publishes no benchmark figures, and never identifies the GPU hardware behind the 20.7-fps number.
Originally reported by huggingface.co
Read the original article →Original headline: Kyutai Releases AURA, an Open-Weights Streaming Avatar Model That Runs at 20.7 FPS on a Single GPU