CoopEval shows contracts beat repetition for LLM cooperation

There's a quiet finding buried in the game-theory-meets-LLM literature that the better language models get at reasoning, the less they cooperate in mixed-motive settings. A new benchmark called CoopEval, being presented at ICML 2026, tries to work out what actually pulls them back toward cooperation.

The authors, from Carnegie Mellon, the University of Toronto and Vector Institute, ETH Zurich, and the Max Planck Institute, put LLM agents through four distinct social dilemmas and layer on four classical mechanisms that theory says should sustain cooperation among rational players: repeated gameplay, reputation systems, third-party mediation, and contract agreements with outcome-conditional payments. On a normalized scale where zero is everyone defects and one is full cooperation, contracting scored 0.801 and mediation scored 0.695. Repetition landed at 0.587. Reputation, in both its weak and strong variants, came in below repetition at 0.321 and 0.227, and the no-mechanism baseline sat at 0.072. So the classic 'they'll play again next round and reputations will build' story is the weakest of the enforceable options here.

The wrinkle the authors call out is that repetition's benefit deteriorates when co-players change, which is a fairly realistic setting given that real agent deployments aren't going to sit across from the same counterpart forever. On the other hand, the paper reports that cooperation mechanisms strengthen under higher optimization pressure, which suggests the wins aren't a quirk of one prompt configuration.

The honest caveat is that this is a poster-page summary rather than the full paper: the project page names only two of the four social dilemmas (a public goods game and a trust game) and does not spell out which LLMs were tested, so it's hard to know how far the ranking generalizes. The takeaway for anyone wiring up multi-agent systems is that if you want cooperative behavior between capable models, the reporting here suggests the useful question is less 'can they learn to trust each other over time' and more 'what contract or mediator sits between them'.