Paper ties 83% of agent failures to compaction summaries

Long-horizon LLM agents are failing because their own summaries are telling them things that never happened. In a new arxiv paper, Guanghui Min and Chen Chen report that in a widely used structured compressor, "83% of failed runs trace back to an action that carries out a specific summary entry, most of which no environment output supports."

The authors frame it as a provenance problem. Compaction prompts ask the compressor to produce decisions and next steps alongside the record of what happened, so each summary mixes environmental facts with the compressor's own reasoning, and the agent cannot tell the two apart later. Their fix, Source-Judged Compaction, is "a training-free post-processing step that scores each summary entry for whether it has a source in the environment outputs or the previous summary, and removes the entries that do not."

On AppWorld and OfficeBench with two agent models, SJC "improves cross-run reliability over the underlying compressor by 2.1%-8.9% without increasing the agent's token usage." The abstract does not name the structured compressor it analyzes, nor which two agent models carry the reliability numbers.