Cian O'Donnell

arxiv.org/abs/2605.26099 Language Models Need Sleep "attention mechanism scales poorly with context length. To handle this, we study a sleep-like consolidation mechanism in which a model periodically converts recent context into persistent fast weights before clearing its key-…

[2605.26099] Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference arxiv.org View on Bluesky →