CLBench-V: best model scores 0.2847 across 3,443 instances

A new benchmark from the CLBench-V team lands in an awkward spot for the multimodal frontier. Six recent multimodal models were run across 3,443 instances designed to test how well they learn from task-specific multimodal context, according to the paper on arxiv, and the best overall score is 0.2847.

The framing matters as much as the number. The authors argue that existing evaluations of context learning have mainly focused on textual contexts, while real-world tasks pull context from figures, tables, converted reports, maps, scenes, and web pages. To probe that gap, CLBench-V organizes tasks around three dimensions: context grounding, new information application, and new knowledge learning. The datasets are a mix of converted public benchmarks and newly constructed sets spanning science, finance, long-document understanding, spatial reasoning, and web-based visual question answering.

No single model dominated. InternVL3.5-30B-A3B was the top performer on context grounding and new knowledge learning, while Qwen3.5-Plus led on new information application. The authors' own reading is that "multimodal context learning remains far from saturated," which is a polite way of saying the field is further from solved than flashier leaderboard scores would suggest.

The honest caveat is that any benchmark result rides on the choices baked into the benchmark. The paper leans on automated construction and filtering procedures to keep new-dataset costs down, and it analyzes judge reliability, context length, image count, and representative failure cases; the abstract does not spell out how much those choices shape the ceiling, and does not report scores from the largest closed frontier systems beyond the six models actually tested. Read 0.2847 as the ceiling among the models CLBench-V ran, not a universal cap.

For anyone shipping VLM features that assume the model can pull answers out of user-supplied documents, scenes, or dashboards, the takeaway is worth internalizing before the next demo: pretraining recall and inference-time context use are not the same skill, and one is much further along than the other.