Ovis-Embedding: one shared backbone tops five omni-modal suites
TL;DR
- Ovis-Embedding uses one Qwen2.5-Omni Thinker backbone to encode text, image, video, and audio in a common representation space rather than separate retrieval towers.
- The authors report state-of-the-art results on MMEB-v3, MMEB-v2, MVEB, MAEB, and RTEB, though the abstract publishes no per-benchmark numbers.
- Native embedding width is 2048 with a post-hoc elastic-dimension module that trims to 1024, 512, 256, or 128 dimensions for tighter deployments.
A new preprint from the Ovis-Embedding Team proposes a single encoder that handles text, image, video, and audio in one representation space rather than gluing together modality-specific towers. Posted to arXiv on September 21, the paper claims "state-of-the-art performance" on five evaluation suites: MMEB-v3, MMEB-v2, MVEB, MAEB, and RTEB.
The system starts from Qwen2.5-Omni Thinker and adds contrastive training on top, using what the authors describe as "a shared multimodal backbone to encode different modalities in a common representation space." No separate retrieval tower per modality; the tokens go into one stack.
Two engineering choices stand out. "The native output width is 2048 because no embedding projection head is added to the backbone," and a post-hoc elastic-dimension module lets deployments trim to 1024, 512, 256, or 128 dimensions. The training recipe leans on focal loss and similarity-based embedding distillation, plus homogeneous-source sampling across the media types.
The abstract publishes no per-benchmark accuracy numbers. MMEB-v3, the largest of the five suites, is scored as the unweighted average over 190 datasets using Hit@1 for image, video, audio, and agent tasks and nDCG@5 for text and visual-document retrieval. A 3B variant, Ovis-Omni-Embedding-3B, has been posted on Hugging Face.
Originally reported by paper
Read the original article →Original headline: Ovis-Embedding Claims SOTA Across Five Omni-Modal Retrieval Benchmarks With One Shared Backbone