Drop-In Triton Kernel Cuts Relative-Position Attention Peak Memory 128× at 8K Tokens

Relative-position attention outperforms RoPE on audio, music, and other structure-sensitive domains but has been impractical at scale due to pairwise offset-dependent computation. This paper makes it competitive with FlashAttention, removing a key barrier for long-context audio and scientific models without changing the attention semantics.