SpecMRL from Renmin and Lenovo cuts MRL embeddings in half
TL;DR
- SpecMRL outperforms vanilla MRL at every reduced dimension on MTEB, with gains up to 3.80 points under 16x compression.
- The method matches full-dimensional MRL using half the embedding size and reports a retrieval speedup of up to 1.86x.
- SpecMRL swaps MRL's fixed coordinate ordering for a learned spectral basis derived from a projection matrix's Gram decomposition via power iteration.
A new embedding paper says you can halve your vector dimensions and still beat the full-size baseline. The claim comes from SpecMRL, a redesign of Matryoshka Representation Learning posted on October 20 by researchers at Renmin University of China, Wuhan University, and Lenovo Group.
Standard MRL takes the first m coordinates of a full representation and hopes earlier dimensions carry more information. SpecMRL replaces that heuristic with a learned spectral basis derived from a projection matrix's Gram decomposition, approximated via power iteration to dodge the numerical instability of standard eigen solvers.
The paper reports SpecMRL 'outperforms MRL at all evaluated reduced dimensions, with larger gains under more aggressive compression,' with gains of up to 3.80 points under 16x compression on MTEB. It also claims the method 'surpass[es] full-dimensional MRL with only half the embedding dimensions,' and reports a retrieval speedup of up to 1.86x.
The abstract publishes no per-task MTEB numbers, and the main hyperparameter sweep sits on Qwen3-0.6B with the projection matrix over-parameterised at R=128D. Code is at github.com/CentreChen/spec-mrl. It lands alongside a dozen other retrieval papers in our RAG feed over the past ninety days.
Originally reported by huggingface.co
Read the original article →Original headline: Renmin and Lenovo's SpecMRL Reworks Matryoshka Embeddings in the Parametric Spectrum