Allen AI's XYBench shows top LLMs answer the wrong question

Across 8,115 queries built to contain a buried misconception, every frontier model Allen AI tested answered the stated question more often than the underlying one. On Stack Overflow prompts, "stated-request solutions appeared in 75–86% of responses, compared with 59–71% for underlying-goal solutions."

The benchmark is called XYBench, and it took the Best Resource Award at EMNLP 2026. It is built on what the authors call the XY problem. The README gives the canonical case: "a user might ask 'How do I parse XML with regex?' when their underlying goal is simply to extract data from XML." Each query carries a stated request (S-request) and an intended request (I-request), and responses are graded on three rubrics: whether the S-solution is present, whether the I-solution is present, and whether the S-I misconception is identified.

The five frontier models evaluated were Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro. In a write-up on Hugging Face, the team reports: "On Stack Overflow, models focused primarily on the underlying goal in only 35–41% of their responses, and on the stated request in 47–57%." They add: "Unlike human reference answers, all five models were less likely to steer users away from that approach and toward a solution suited to their goal."

There is a visible gap between recognizing the goal and writing to it. When the models were given a multiple-choice pick between a stated-request answer and a goal-focused one, they "chose the goal-focused answer in 71–84% of cases" on Stack Overflow, well above the 35–41% rate they hit writing freely. Two researchers on our Who's Who radar had flagged the paper by the time we picked it up.