arxiv.org web signal

MEMOIR adds 11.92pt evidence recall to LLM agent memory

Agents RAG ai-business

TL;DR

  • MEMOIR lifts evidence recall@5 by 11.92 percentage points over bare cosine retrieval on 1,905 deduplicated LoCoMo QA pairs.
  • Confidence-gated feedback adds another 1.50 points without reference answers or reranker training; a deployment-unavailable oracle adds 8.05 points on multi-hop.
  • Judge-gated merging shifts text-evidence recall from 58.82% to 59.08%, shrinks the active store 1.42%, and logs zero losses across 100 audited merges.

On 1,905 deduplicated LoCoMo QA pairs, MEMOIR raises evidence recall@5 by 11.92 percentage points over bare cosine similarity. That is the headline number from an arxiv preprint by Mayank Anand, Subham Raj, Aman Vaibhav Jha and Sriparna Saha proposing a training-free memory lifecycle for long-horizon LLM agents.

The authors describe the failure mode bluntly. "Existing systems treat these as separate problems: memories are written prospectively with static importance scores that are never revised after use," the abstract reads. "This creates two coupled failure modes: consolidation can silently destroy evidence that future queries require, while retrieval continues to favor stale memories."

Their fix reduces to "one principle: attribute usefulness after use, and screen for preservation before compression." Before any merge, screening gates test whether answer-relevant evidence is preserved; after each query, attribution updates the retrieval priority of whichever memories supported the answer. A UCB-style exploration bonus keeps under-exposed memories in play.

The numbers underneath the headline are smaller. Confidence-gated feedback, the piece that runs without reference answers or reranker training, adds 1.50 points on top of retrieval. An oracle variant the authors flag as "deployment-unavailable" adds 1.84 points overall and 8.05 points on multi-hop. Judge-gated merging shifts text-evidence recall from 58.82% to 59.08% while reducing the active store by 1.42%, with "0 observed losses in 100 independently audited merges" and a one-sided 95% upper bound on loss rate of 2.95%.

The paper lands in a run of memory-and-RAG agent work we have been tracking, where retrieval failures rather than model size increasingly bound long-horizon agent performance.