paper web signal

GAFU taps UMM generation side for fine-grained vision gains

TL;DR

  • GAFU is a post-training framework that compresses generation-side representations into "G tokens" to feed the understanding pathway of unified multimodal models.
  • The paper reports maximum relative improvements of 20.00% on VStar-Bench, 17.94% on OmniSpatial, and 8.62% on VSR-Bench over same-data SFT baselines.
  • The authors argue generation-side features catch fine-grained semantic changes that understanding-side representations miss, if spatial organization is preserved.

A new preprint argues the generation side of unified multimodal models carries fine-grained visual information the understanding side is leaving on the table, and that a post-training wrapper can move it across.

The framework, called GAFU, "compresses generation-side visual representations into compact G tokens that preserve fine-grained visual details, enabling task-conditioned retrieval by the understanding pathway," the arxiv paper reports. On three fine-grained vision benchmarks, the authors report maximum relative improvements of 20.00% on VStar-Bench, 17.94% on OmniSpatial, and 8.62% on VSR-Bench, measured against supervised fine-tuning baselines trained on the same data.

The underlying claim is that generation-side features "are more sensitive to fine-grained semantic changes, capture task-relevant changes missed by understanding-side representations, and depend on correct spatial organization for this advantage." The abstract gives those numbers as maxima, not averages, and does not name the specific UMM architectures tested.