PMD distills training-time procedural memory into LLM weights

Reinforcement learning with verifiable rewards grades each rollout, updates the policy, and throws the rest away. A new paper on arXiv argues that is where the information is.

The authors call their method Procedural Memory Distillation, or PMD. "Across episodes and epochs, the model repeatedly encounters related problems under a changing policy, producing cross-episode signals that episode-local updates cannot capture," they write: which strategies pass verification, which failure modes persist, which patterns recur. PMD keeps those signals in an online store organized at three levels of abstraction: raw trajectories, self-reflected strategies and lessons, and higher-level behavioral patterns that recur across problems. A "memory-conditioned self-teacher" then supervises the student on its own rollouts, pushing procedural knowledge into the policy's weights so the final model ships memory-free at inference.

On Qwen3-8B and OLMo3-Instruct-7B, PMD improves over SDPO by 3.8-5.5% on SCIKNOWEVAL and 7.9-13.6% on LIVECODEBENCH. The whole design hinges on what the paper calls co-evolution: "the policy generates rollouts that update the memory, and memory shapes the supervision that updates the policy." Freezing either side trails the full method by more than 10% across SCIKNOWEVAL domains.

The abstract publishes no wall-clock or compute figures for the memory scaffolding, and the gains are shown on two 7-8B bases and two benchmark families. Two of the researchers we track circulated it the day it landed.