NVLabs' Sol-Attn Speeds Wan 2.1, HunyuanVideo 2x Training-Free

Attention cost has been the ugly secret of video diffusion for a year now. Every extra frame lengthens the token sequence, and dense attention scales quadratically, so a longer clip is not just a longer render, it is a disproportionately more expensive one. A new paper on Hugging Face from a group including Song Han proposes Sol-Attn, a training-free way to make that attention step considerably cheaper on the models people are actually running.

The reported numbers are the eye-catching part. Standalone, Sol-Attn claims a 2.02x end-to-end speedup on Wan 2.1-14B and 2.12x on HunyuanVideo-13B against dense FlashAttention-3, and up to 2.4x on LTX 2.3. Combined with a companion kernel-fusion and caching layer called Sol-Engine, the authors push that to 3.48x on Wan 2.1-14B and 5.08x on HunyuanVideo-13B. The code sits inside the NVLabs Sana repo, which suggests the Sana team is treating this as a shared inference substrate rather than a one-off release.

Mechanically, the interesting choice is doing block selection and approximation inside a single online-softmax pass. Prior sparse-attention methods either fix a sparsity budget in advance or spend real compute materializing proxy score maps, and they simply drop the blocks they do not pick. Sol-Attn thresholds blocks on the fly and reuses the proxy scores of the discarded blocks to approximate their contribution rather than throwing them out. That is the pitch for why it should hold up under aggressive sparsity where earlier methods visibly degrade.

The honest caveat is that these are the authors' own benchmarks against FlashAttention-3, and the paper frames itself around a 'quality-efficiency frontier' rather than lossless acceleration; the community will need to sanity-check visual quality at long clip lengths and see how the numbers survive on hardware other than the reference setup, since the project page notes that a B200 kernel is still under development. What the writeup does not spell out is memory footprint or how the speedup scales with clip length, which is what serving teams will actually want to know.

If it holds up, the practical read is straightforward. Anyone hosting Wan or HunyuanVideo for text-to-video, and anyone benchmarking open video generation against Sora and Runway on cost, has a new number to plug into the spreadsheet before the next GPU contract goes out.