UW-Meta paper: language models that edit their own context

A new paper from a team at the University of Washington and Meta Superintelligence Labs argues that the right place to manage an LLM's context is inside the LLM itself.

The idea, laid out in the Context Language Models repository and the matching arXiv preprint, is to treat the model's context as a plain file the model can edit however it wants. "We introduce Context Language Models (CLMs), language models that natively manage their own context," the abstract reads. "We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file."

The authors (Rulin Shao, Shannon Zejiang Shen, Pang Wei Koh, and ten co-authors across UW, Meta, MIT and Trillium Labs) report that building CLMs zero-shot with existing models beats prior context-management strategies by "11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task."

They also report that an online RL method lifts Qwen3.5-9B on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs, and that a co-designed Suffix Cache Reuse serving layer cuts server-side compute by 35% relative to standard SGLang at matched performance.

All of those numbers come from the paper itself; no third-party replication yet. The code is released under CC BY-NC 4.0, and two researchers we track had circulated the repo within a day of it going up.