Experience Distillation Keeps 64.8% of Agent ICL Gains in Weights

A quiet result worth flagging for anyone who ships agents into environments that cost real money to sample. In a new arXiv paper, Chenhui Gou and co-authors propose Experience Distillation, a method that takes the gains an agent gets from in-context learning on its own rollouts and pushes them into the model weights, with no additional environment interaction.

The headline number is that the technique retains at least 64.8% of the in-context learning gains across the tested domains. For comparison, the authors report that direct supervised fine-tuning on the same experience recovers only 3.8%. That gap is the interesting part. In-context learning has always been the fastest way to make an agent better at a specific setup, but the improvement evaporates the moment the context window is cleared or a new session starts. If most of that uplift can actually be baked into the weights, the operational picture for long-running agents changes.

Why that matters if you are not training models yourself: environment interactions are the expensive part of agent development. The paper claims that Experience Distillation combined with in-context learning matches a classical RL baseline with at least 9.6 times fewer environment samples. For teams working on software engineering agents, browser agents, or anything touching a real sandbox, sample count maps almost directly to dollars.

The honest caveat is that the evaluation is narrow. The paper reports results on 749 curated software-engineering tasks and six text-adventure games, which is a defensible testbed but not proof that the same 64.8% floor holds for robotics, tool-using web agents, or safety-sensitive deployments. What the reporting does not give you is the compute cost of the distillation step itself, the base model used, or how this stacks up against memory and retrieval approaches that keep the context around rather than collapse it.

Still, the direction is the part worth watching. If per-user or per-deployment experience can be turned into durable weight updates without another round of rollouts, the teams that benefit first are the ones paying the most per environment step today.