InMind Benchmark Exposes 14.4% Ceiling in Agent Memory Recall

A quiet but pointed result landed on arXiv this week that anyone building on agent memory should read before their next roadmap review. In Keep It InMind, researchers from USTC introduce a benchmark of 125 tasks across ten life domains and report that six vector, graph, and agentic memory systems retrieve the right stored fact at most 14.4 percent of the time when the user's query shares no textual overlap with what was written to memory. The same systems hit up to 100 percent recall when you ask them directly for the fact, and 84.0 percent when the memory is already sitting in the context window.

The gap between those numbers is the whole story. It says the bottleneck is not what these systems remember or how they encode it, but the query-conditioned interface that decides which memory to fetch. The authors tested the obvious lever, cranking embedding dimensionality up eight times, and report the gap is left "essentially intact." More capacity does not help if the query itself does not lexically resemble the memory you need.

Why this matters if you are shipping an agent product: the pitch for long-term memory is usually that the assistant will quietly recall a preference or an earlier decision when it becomes relevant. The InMind result is a structural argument that today's production memory stacks only reliably do that when the user rephrases their old wording. In practice, users do not. They ask about the outcome, not the fact they stored.

The honest caveat is that the paper does not name the six systems it benchmarked, so you cannot yet check whether your specific vendor is in the sample, and the abstract does not break performance down by domain. What the reporting also does not give you is a picture of how their proposed fix, a diagnostic probe that keeps memory visible before the query arrives, scales when memory stores get large.

The forward-looking read is that routing and selective visibility, rather than bigger embeddings or fancier graphs, are where the next useful work in this area probably lives, and the teams that solve it first have a clean, benchmark-backed wedge.