AB-Attention Skips 89-92% of Attention on Wan2.1, HunyuanVideo

A new preprint on Hugging Face Papers, "AB-Attention: Accurate Block Attention for Accelerating Video Diffusion Transformers," claims a training-free kernel that skips 89-92% of attention compute on Wan2.1 and HunyuanVideo while staying closest to dense attention in full-video PSNR. On a single H20 GPU the authors report 1.84-2.23x end-to-end speedup, and PSNR gains of 1.6-4.4 dB over the strongest prior block-sparse baselines, including SVG2 and DFSAttn.

The design has three moving parts. An Intra-Block Clustering Selector (IBCS) scores key/value blocks "with lightweight query clusters formed only within each query block, avoiding costly global clustering and token permutation." Sampling with Inclusion-Probability Weighting (SIP) then "samples blocks in proportion to these scores and reweights them by inverse inclusion probability," so that, as the paper puts it, "one block-sparse pass estimates dense attention rather than truncating it." A custom kernel gathers key/value blocks much smaller than a tile into full tensor-core tiles, so finer selection does not cost tensor-core efficiency.

The authors also quantify why finer blocks help: shrinking key/value blocks from 128 to 4 tokens raises the attention mass retained by oracle top-k from 59% to 69% at 89.16% sparsity. Where competing baselines compute 1.8-3.6x as much attention as AB-Attention at matched settings, the paper reports its approach still improves PSNR by "up to 4.4 dB over the strongest baseline."

AB-Attention is published with code at github.com/InfRec-Cardinal/AB-Attention. It joins a run of inference-efficiency work on our radar this week, following SpecSlot's pixel-space generative transformer scheduler tracked a day earlier.