Zero-shot 27B VLM ties supervised 8B reranker on visual docs

Reranking the top sixteen pages returned by a visual-document retriever, a general 27B vision-language model, prompted once and never trained for retrieval, scored 84.79 nDCG@5 on MMEB-V2 visual documents. A supervised 8B reranker trained for the job scored 84.78.

The paper, Multimodal Reranking Is a System One Decision, reframes the reranker as a decision taken by a general VLM: label the top-K candidates A to P, feed query and candidates into one prompt, run a single forward pass, and read the next-token distribution restricted to the label tokens. The renormalized probabilities become the new ranks. The backbone is Qwen3.8-27B. Candidates below rank 16 keep the first stage's order.

Across three benchmarks, all reranking the top 16 of Qwen3-VL-Embedding-2B, Qwen3.8-27B zero-shot beats the supervised Qwen3-VL-Reranker-2B by 2.3 hit@1 on MMEB-V2 image retrieval, 2.1 nDCG@5 on visual documents, and 1.8 nDCG@10 on ViDoRe V3. On visual documents it matches Reranker-8B. On natural images Reranker-8B still leads by 1.6 hit@1. A decision-tuned variant on the same backbone, JEV-27B-VL, which adds a LoRA adapter and a decision head trained only on text decisions, tops both document benchmarks at 85.42 and 62.03.

The authors frame the deployment case plainly: "retrieval-augmented systems often already serve a large VLM to generate answers. If that model can rerank through one decision over the candidates, the system needs no separate reranker to train, serve and keep up to date."

They are candid about the confound. The 27B model is larger and from a newer generation than the 2B and 8B rerankers it is compared to, so the result shows "a zero-shot VLM can match a dedicated 8B reranker on MMEB-V2 visual documents today, not that it would match a dedicated reranker trained from the same backbone." Latency on one H200 is 3.97 seconds per query, roughly 1.1 times the released-code latency of Reranker-8B and 1.5 times its batched version. Smaller Gemma 4 variants (E2B and E4B) lower the first stage on all three benchmarks, with the readout only starting to help documents from 12B upward. The code is released. The paper lands amid an active cycle of multimodal retrieval work, more than 90 stories in our tracker over the last 90 days, including yesterday's note on SpecMRL halving MRL embeddings.