APM-Bench Tests Egocentric Video AI's Memory Across Sessions

A team led out of Shanghai Jiao Tong University and Eastern Institute of Technology Ningbo has released APM-Bench, a benchmark that reformulates streaming egocentric video as multi-session life trajectories — 549 sessions across 104 trajectories, evaluated against 2,719 candidate items — and uses that structure to test whether always-on assistants can actually retrieve what they saw when the user comes back later. The premise on the paper's Hugging Face page: "To serve as real-world personal assistants, streaming video models need persistent memory that retains past experiences for later use."

Sessions run about 13 minutes on average, full trajectories about 69, with realistic time gaps in between. Twelve tasks are grouped into three capability families: Cross-session Understanding (episodic recall, entity state tracking, temporal reasoning), Real-time Perception (action recognition, counting, OCR, spatial understanding), and Adaptive Response (evidence-ready answering, registered-condition response, memory-grounded proactive assistance, proactive reminder, task progress guidance). Inter-annotator agreement across those tasks is reported at a Cohen's kappa of 0.868.

The headline result across the memory protocols the authors evaluate is a trilemma. "Existing methods still struggle to simultaneously achieve reliable long-term recall, low overhead, and effective proactive assistance across sessions," the paper reports, framing the design space as an explicit trade-off between utility, latency, and storage.

It lands in the same week we tracked InfiniHand, another egocentric streaming system, and adds a scoreboard to a space — 67 multimodal stories in the last 90 days on our tracker — that has been long on demos and short on shared evals.