Found first: a primary source the press has not covered yet.
A team including Quanquan Gu (UCLA), Mengdi Wang (Princeton), and Turing Award laureate Andrew Chi-Chih Yao (Tsinghua) has published a theoretical analysis of recurrent fast-weight memories and selective state-space models, deriving the online learning rules underlying these architectures. The paper, submitted August 27, 2026, shows what objective each popular recurrent state update is actually optimizing.
What the source says
The authors study fast-weight and SSM state transitions under read-after-write autoregressive semantics, treating each state update as a step of online learning on a prefix-aligned example. Deriving normalized first-order updates for squared-error regression and negative inner-product objectives, they introduce six variants: Falcon-1 (a scalar NLMS update), Falcon-2 (a per-column extension), and Falcon-3 (a sliding-window mini-batch update), alongside Falcon-1A, Falcon-2A, and Falcon-3A as the corresponding inner-product counterparts. Representative variants remain competitive in language modeling and improve length extrapolation on variable-digit addition tasks. The framework explicitly separates temporal alignment, plasticity, forgetting, and bounded rehearsal in recurrent sequence models.
Why it matters
Mamba-class and fast-weight architectures have proliferated without a unified account of what their state updates are mathematically optimizing. This paper supplies that account, mapping each recurrent update rule to a specific online regression objective with known properties. Practitioners gain a principled basis for choosing or designing state updates with explicit control over forgetting and plasticity, rather than treating them as empirical hyperparameters. Yao's involvement signals that foundational theorists are now directly engaging with the mechanisms behind modern sequence models.