Adversarial Policy Beats KataGo Over 97% at Superhuman Play

For a decade the go-to example of 'superhuman AI' has been game-playing systems that no top human can touch, and Go was the flagship case. A paper by Tony T. Wang, Adam Gleave, Stuart Russell and collaborators, published on arXiv and accepted to ICML 2023, punches a fairly large hole in that story. The authors trained an adversarial policy that reportedly wins more than 97% of games against KataGo, one of the strongest publicly available Go engines, when KataGo is running at superhuman settings.

The interesting part is how. The adversary does not out-play KataGo in the ordinary sense. It exploits a blind spot, tricking the engine into what the paper describes as serious blunders. The strategy is structured enough that a human expert can execute it by hand and beat superhuman Go AIs without any computer help. And it is not a fluke of one system: the authors report that the attack transfers zero-shot to other superhuman Go-playing AIs.

The obvious defensive move, adversarially training KataGo against the exploit, did not close the gap. According to the paper, the vulnerability persists even when KataGo agents are adversarially trained to defend against the attack, which suggests something structural rather than a bug to be patched over a weekend.

Why this matters outside of Go: the mental model that 'superhuman on the benchmark' implies 'robust in the wild' gets a lot weaker if a small, targeted policy can produce >97% wins against a system that was considered a settled reference for AI capability. The same class of failure could plausibly show up in any reinforcement-learning system that is graded on an average-case performance metric.

The honest caveats are the source's own. This is one attack, on one game, from one research group; take the specifics as reported, not settled. What the paper does not tell you is which currently deployed commercial or research systems are exposed to the same class of exploit, or exactly why adversarial training failed to close the gap. The forward-looking read is that adversarial red-teaming of 'superhuman' claims is now something the field probably has to do routinely, not as a curiosity.