Select, Compress, Reinvest: OMP holds own vs. new selectors

Eight query-selected frames beat sixteen uniformly spaced ones by 6.9 points on LongVideoBench's hour-long bin, the largest single lever in a controlled study of how long-video multimodal language models spend their visual-token budget.

The arXiv paper, 'Select, Compress, Reinvest' by Prakhar Khatri, holds the frame scorer, prompt boundary, resolution policy and answering model fixed and varies one decision at a time across six training-free selection rules, three long-video benchmarks and two answering models. 'Which frames survive that slice is usually treated as a preprocessing detail; we test whether it should be,' the paper says.

Selection dominates. Orthogonal Matching Pursuit, described as 'an unmodified decades-old sparse-approximation algorithm,' matches or comes within a point of every purpose-built selector across all three benchmarks. Spatial compression is close to free: halving each frame's spatial budget at fixed timestamps costs at most 0.44 points. The payoff comes from reinvestment: spending those freed tokens on twice as many compressed frames returns a further two to three points, and compression only pays off once its savings are spent this way.

Khatri also flags a 0.07 to 3.74 point gap between two harnesses running the same published rules at the same budget, plus an implementation bug in his own AKS baseline. The result lands amid a steady stream of long-video and multimodal work tracked on our multimodal feed, where LatentStream recently traded retrieval for an evolving latent memory over long clips.