Paper proposes 'Sleep' paradigm for LLMs to consolidate memory

Current language models cannot carry what they learn in-context into their long-term parameters. That is the premise of a new paper on arXiv by Ali Behrouz, Farnoosh Hashemi, Adel Javanmard and Vahab Mirrokni, whose proposed fix is to let the models sleep.

The authors call it a 'Sleep' paradigm with two stages. The first, Memory Consolidation, is 'an upward distillation process, called Knowledge Seeding, where the memories of a smaller-self are distilled into a larger network to provide more capacity while preserving the knowledge.' The method combines on-policy distillation with reinforcement-learning-based imitation. The second stage, Dreaming, is a self-improvement phase in which 'the model uses RL to generate a curriculum of synthetic data to rehearse new knowledge and refine existing capabilities without human supervision.'

The paper reports experiments on 'long-horizon, continual learning, knowledge incorporation, and few-shot generalization tasks' that 'support the importance of the sleep stage.' The abstract itself publishes no per-task numbers. The link was moving through a couple of the researcher feeds we track by the time our scanner logged it.