GAFU taps UMM generation side for fine-grained vision gains
TL;DR
- GAFU is a post-training framework that compresses generation-side representations into "G tokens" to feed the understanding pathway of unified multimodal models.
- The paper reports maximum relative improvements of 20.00% on VStar-Bench, 17.94% on OmniSpatial, and 8.62% on VSR-Bench over same-data SFT baselines.
- The authors argue generation-side features catch fine-grained semantic changes that understanding-side representations miss, if spatial organization is preserved.
A new preprint argues the generation side of unified multimodal models carries fine-grained visual information the understanding side is leaving on the table, and that a post-training wrapper can move it across.
The framework, called GAFU, "compresses generation-side visual representations into compact G tokens that preserve fine-grained visual details, enabling task-conditioned retrieval by the understanding pathway," the arxiv paper reports. On three fine-grained vision benchmarks, the authors report maximum relative improvements of 20.00% on VStar-Bench, 17.94% on OmniSpatial, and 8.62% on VSR-Bench, measured against supervised fine-tuning baselines trained on the same data.
The underlying claim is that generation-side features "are more sensitive to fine-grained semantic changes, capture task-relevant changes missed by understanding-side representations, and depend on correct spatial organization for this advantage." The abstract gives those numbers as maxima, not averages, and does not name the specific UMM architectures tested.
Originally reported by paper
Read the original article →Original headline: GAFU Unlocks 20% Fine-Grained VLM Gains From Hidden Generation Pathways