ByteDance Seed's Cortex Bakes Context Into LLM Weights, Keeps 99.8% Base Performance After 50 Edits
Summary
ByteDance Seed's Cortex paper introduces Memory Consolidation, a framework that distills context-induced gains into low-rank or structural feed-forward network updates while freezing base weights. On a 14B model, Cortex-Structural recovers 88.9% of context gains after 50 sequential factual edits while retaining 99.8% of non-target performance; evaluations span 4B–120B dense and MoE backbones across text and multimodal.
Originally reported by huggingface.co
Read the original article →Original headline: ByteDance Seed's Cortex Bakes Context Into LLM Weights, Keeps 99.8% Base Performance After 50 Edits