AMIE team: benchmarks alone can't earn clinical AI trust

Trust in clinical AI, the authors of a new Nature Medicine comment write, "cannot be benchmarked into existence. It must be earned through rigorous prospective studies in real-world clinical settings."

The piece is by Mike Schaekermann of Google Research, Anil Palepu, Adam Rodman and colleagues at Google DeepMind, Harvard Medical School, Stanford and Beth Israel Deaconess Medical Center. It sits alongside the team's own companion feasibility trial of AMIE, the Articulate Medical Intelligence Explorer.

That companion study ran 100 adult patients through AMIE for pre-visit history taking at BIDMC. Human safety supervisors monitored every patient-AMIE session in real time and, in the trial team's words, "did not need to intervene." AMIE's differential included each patient's final diagnosis in roughly 90% of cases.

The comment's sharper claim is that what these studies expose is rarely the model. "The hardest lessons often concern the humans and systems around the AI, not the technology itself," the authors write. Two researchers we follow had it in circulation the day it went live.