CORE distills MLLM reranker rankings into embedding models
TL;DR
- CORE-RERANKER-8B hits an 82.7% total average across COLA, SUGARCREPE++ and NEGBENCH, outperforming Jina-Reranker by 10.7 points.
- The distilled embedding, CORE-EMBED-8B, posts the best total average at 0.666 among the embedding models evaluated.
- Listwise Rank-KL distillation beats contrastive learning and pairwise CoSENT under matched data and tuning budgets.
MLLM-based embedding models keep failing on compositional retrieval, unable to tell apart scenes that share the same concepts but bind different attributes to them. In CORE, Tingyu Song and coauthors argue the fix is already inside the backbone: "the same backbone can resolve such distinctions when used as a cross-attentive reranker," the paper states, so the authors distill that reranker signal back into the embedding path.
Their method synthesizes candidate lists across five compositional matching levels and trains the embedding with a Rank-KL objective that reproduces the reranker's ordering. They pit that loss against contrastive learning and pairwise CoSENT under matched data and tuning budgets, and report Rank-KL comes out strongest.
On COLA, SUGARCREPE++ and NEGBENCH, CORE-RERANKER-8B lands at an 82.7% total average, 10.7 points over Jina-Reranker, while CORE-EMBED-8B posts the best embedding average at 0.666. The paper reports the improvements "transfer to the MCMR benchmark without sacrificing retrieval performance on COCO and Flickr30K."
Originally reported by paper
Read the original article →Original headline: CORE Distills Reranker Knowledge Into MLLM Embeddings, Gains 10.7 Points on Compositional Tasks