arxiv.org web signal

TsuGO uses Go life-and-death puzzles to grade LLM search

TL;DR

  • TsuGO scores how LLMs organize a search on Go tsumego problems, not just whether they land on the correct final move.
  • The paper reports most models behave 'much closer to unguided search algorithms than to neural-guided KataGo' on these puzzles.
  • Longer chain-of-thought and higher token efficiency do not necessarily produce better search; stronger models commit to the right candidate earlier.

A new arXiv paper argues that most large language models, when asked to reason through Go life-and-death puzzles, behave less like the neural-guided search engines that mastered the game and more like brute-force explorers. The benchmark, called TsuGO and posted to arXiv on August 13, 2026 by a team including Shunwen Bai, Ziping Ma and Qingpei Guo, leans on tsumego because, in the authors' words, they "provide closed and verifiable solution spaces with an inherent adversarial structure, making candidate generation, response checking, branch comparison, and backtracking necessary."

That closed structure is the point. The authors want to strip away the domain-knowledge advantage that clutters most reasoning benchmarks, so they can look directly at how a model organises a search: which candidates it generates, which branches it prunes, how quickly it locks onto the winning move. By parsing chain-of-thought into a structured search tree, TsuGO reports a Search Efficiency metric alongside token counts, rather than only scoring the final answer.

The headline finding is uncomfortable for the "just add tokens" school of reasoning. The paper says longer CoT and higher token efficiency do not necessarily mean better search, and that stronger models succeed by "finding the correct candidate earlier and sustaining effort on productive branches" rather than by exploring more. Most models still operate "much closer to unguided search algorithms than to neural-guided KataGo."

The public abstract stops short of specifics: no model names, no per-model efficiency numbers, no dataset size, so anyone wanting a scoreboard will have to open the PDF. The framing also stays narrow to Go, and the authors leave open whether their Search Efficiency metric survives the jump to code, math, or agentic planning. Two of the experts in our Who's Who directory have shared the paper's link, a small signal it is landing with reasoning researchers.

If the metric does hold up outside tsumego, it hands post-training teams a diagnostic they currently lack: a way to see whether a new reasoning recipe is actually shortening the path to an answer, or just spending more tokens on the same random walk.

Shared on Bluesky by 2 AI experts