SpatialGen-Bench Lets Image Models Answer Spatial Tasks in Pixels

A new paper argues that image-generation models have been graded on the wrong test. Spatial reasoning benchmarks typically demand a text answer or a coordinate, which is fine for a language-first model but forces a pixel-first model into an unnatural output. The paper, posted to arXiv under the title "Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text," proposes a fix: let the image model answer by drawing.

The authors introduce ProVisE, described as a "benchmark-agnostic framework that elicits protocol-constrained visual answers from image-generation models and parses them into structured predictions compatible with original metrics." Alongside it comes SpatialGen-Bench, a diagnostic benchmark of 470 samples across 14 spatial subtasks and four capability levels, with validation drawn from six external spatial benchmarks. The point is to keep the existing evaluation metrics but change the shape the answer arrives in.

The headline result is more careful than the punchier version circulating. When spatial answers can be externalized directly in pixel space, image-generation models are competitive; text-output VLMs still retain a clear advantage in compositional spatial reasoning. That is not "image models match VLMs" so much as "each side is stronger where its output modality is natural," which the authors frame as complementary strengths of pixel-space expression and text-based reasoning.

The honest caveat is that the abstract does not name which specific models were tested or publish per-subtask scores, and a 470-sample diagnostic is a probe, not a final verdict. Whether the ProVisE parsing step itself introduces its own quirks is worth watching, since a framework that turns generated pictures into structured predictions is doing non-trivial interpretation work between the model and the metric.

For teams building embodied or robotic systems, the framing matters more than the ranking. If pixel-native models really are underrated on spatial tasks, the leaderboards they have been losing on are the wrong scoreboard, and hybrid setups that keep the compositional reasoning in text and the spatial answer in pixels look like the direction worth building toward.