GPT and Qwen flip picks when asked what a director sees

Asking a vision-language model what a director can see, before asking which block to hand that director, raises its chance of picking the block the director cannot see by 12 to 88 percentage points. The gap holds across six GPT variants, and runs about 17 points on Qwen3.8-27B, according to a new arxiv preprint from Jiaxuan Guo and colleagues.

The setup is a rendered Director Task. A partition obscures the largest block from a director who asks for "the largest block." The perspective-aware answer is the second-largest, the one the director can actually see. If a benchmark asks the model first which blocks the director sees, and then asks which to hand over, the model mostly reports the perception correctly, then reaches for the hidden block anyway.

The authors' reading of the reasoning traces is blunt: "the question shifts how the request is read, from the director's view toward the model's own." When the prompt instead states up front what the director knows and references, fewer than 1% of responses combine a correct perspective report with a hidden-block pick.

Two researchers in our tracker posted the link within hours, which is unusual circulation for a methodology note. The paper's recommendation is narrow and pointed: perspective-taking benchmarks should run paired prompts, identical except for the perspective question, so a model's theory-of-mind score is not just a measurement of the question that preceded it.