Stanford-Oxford-MIT Benchmark Caps Top AI Teammate at 82.2

A benchmark built by researchers at Stanford, Oxford and MIT reports that the strongest frontier AI works as a teammate only 82.2 out of 100, and that leading models reach the best overall deal in just 14% of conflicts even after asking what the teammate wants 99% of the time.

The Collaborative Intelligence Benchmark puts 18 closed- and open-weight models through 240 tasks across eight dimensions of teamwork, drawn from aviation accident reports, clinical cases, misconduct records and existing coding and negotiation sets. Gemini 4 Argon tops the leaderboard at 82.2 at high reasoning effort and 76.6 at default settings. 'Every model, across sizes and families, has clear weaknesses as a teammate,' the authors write.

The failure mode the paper highlights is not inability but passivity. Frontier models 'cave under interpersonal pressure, reaching the best overall deal in only 14% of conflicts,' per the abstract. In the conflict tasks specifically, those same models find the best joint deal 100% of the time when they see both sides' information, 84% with a cooperative teammate, and 14% with a pushy one; 81% end up accepting a weaker offer despite probing what the other side needs.

The gap carries over to inherited work: pass rates drop from 97% to 81% 'when they take over a task in which a weaker model has cheated.' Across 5,094 failed runs, 45% occurred after the model recognised the problem. Capability leaderboards do not predict teammate scores either, since models 'with nearly the same capability score differ by up to 11 points as teammates.' Pushing reasoning effort higher helps emotional intelligence and initiative on average, but makes some systems give in more in conflicts, with Claude Opus 5.5's rate of accepting weaker offers climbing from 90% to 96%.

The authors are releasing 160 of the 240 tasks publicly, with 80 held out for evaluation and a 40-task lite subset that correlates with the full set at Kendall's τ = 0.95.