ACL 2026: Chain-of-Thought Hurts Multimodal Visual Grounding

Chain-of-thought reasoning has become nearly automatic in modern multimodal deployments, but a paper accepted at ACL 2026 argues that default is wrong for a meaningful class of tasks. Across an evaluation spanning 12 tasks, 14 non-reasoning models, and 8 reasoning models, the researchers found CoT creates "undesirable side effects, such as reduced performance in visual grounding and object counting," the kind of pixel-level perception work that many real-world pipelines depend on.

The paper's title, "Look Light, Think Heavy," names the mechanism the authors identify. During reasoning, "visual reflection consistently diminishes" even as verbal reasoning ramps up. The model's extended chain of thought displaces, rather than supplements, close visual attention. For tasks like locating objects in an image or counting items in a scene, that is a functional regression, not an acceptable trade-off.

CoT does earn its place on tasks requiring mathematical, scientific, or multi-image reasoning, where the paper finds it genuinely helps. The problem the authors diagnose is not CoT itself but applying it uniformly without regard to whether a task is primarily perceptual or inferential.

One limit worth naming is scope: the paper also notes that existing multimodal reasoning models show only "marginal overall improvements, possibly due to an overemphasis on mathematical reasoning," which raises questions about whether training data choices are compounding the problem. The study stops short of reporting precise magnitudes for the performance drops on specific benchmarks, and it leaves open whether closed frontier models outside the evaluation show the same degradation.

For teams building multimodal pipelines, the practical direction is task-aware routing: CoT on demand for reasoning-heavy subtasks, bypassed for perceptual ones. That shift alone could recover accuracy on visual grounding and counting without requiring any model changes.