Aalto Paper: VLM Attribution Bottleneck Is Coordinates, Not Skill

A new paper out of Aalto University, posted on Hugging Face, argues that a big chunk of what we have been calling a vision-language 'grounding' problem is really an interface problem. When you ask a VLM to point at the evidence for its answer by outputting bounding-box coordinates, it mostly fails. When you ask the same model to quote the source passage in plain text and let a retriever snap that quote onto a layout block, it does much better.

The numbers give the argument its bite. Across six open models spanning 8B to 31B parameters (Qwen3-VL-8B, Qwen3-VL-30B-A3B, Qwen3.5-9B, Qwen3.5-27B, Gemma-3-12B, Gemma-4-31B), box recall at IoU 0.5 climbed from ≤8.1% under the coordinate interface to 25.9-46.9% under the quote-and-retrieve interface, and the attribution hallucination rate roughly halved, from 82-97% down to 39-65%. On Qwen3-VL-30B-A3B, only 33.1% of coordinate responses even parsed cleanly, while the same model was quite happy to describe evidence locations verbally by naming page numbers and section titles. The authors then apply a GRPO training recipe that skips region-level labels entirely, using a judge-based reward gated on answer correctness, and push the 8B model's strict attributed accuracy from 22.4% to 33.8%.

If this replicates, it matters because it changes what a citation feature costs to ship. The dominant assumption behind training VLMs to cite has been that you need expensive human-annotated bounding boxes and probably a bigger model. What this paper describes is a pipeline where a plain layout parser (MinerU here) and an embedding model do the geometry, the VLM does language, and reinforcement learning can improve attribution using a judge score instead of a labeler. Teams building document-QA on top of open 8-31B models have a cheaper thing to try before the annotation contract.

The honest caveats are on the paper's own limitations page. It only covers single-document attribution; recall is bounded by the parser, whose ceiling at IoU 0.5 is 88.2%; purely graphical evidence without a caption cannot be quoted and therefore cannot be attributed this way; and hallucinated quotes are snapped to plausible regions rather than rejected, so false-confidence failures do not disappear, they change shape. The evaluation is also entirely on open models, so whether closed frontier VLMs share the same coordinate bottleneck is not something this paper answers.

The forward-looking bit is who this helps. If you were about to budget a region-label pipeline for a legal-review or medical-notes citation feature, the finding to test this week is whether your current open backbone, unchanged, reaches production-usable attribution once you stop asking it for coordinates.