AMIE authors: medical AI trust requires prospective trials

One hundred adult patients text-chatted with Google's conversational diagnostic AI, AMIE, at Beth Israel Deaconess Medical Center's Healthcare Associates clinic in Boston, up to five days before their scheduled appointments. Human safety supervisors watched every session in real time. The trial ran from April to November 2025.

That prospective study is what an author group from Google Research, Google DeepMind, Harvard Medical School, Beth Israel Deaconess, Stanford and Included Health points at in a Nature Medicine commentary arguing that benchmark leaderboards can no longer stand in for clinical evidence.

"Trust in clinical artificial intelligence (AI) cannot be benchmarked into existence," the authors write. "It must be earned through rigorous prospective studies in real-world clinical settings, where the hardest lessons often concern the humans and systems around the AI, not the technology itself."

The companion trial itself produced numbers that are legible to a clinician. AMIE placed the eventual final diagnosis in its top seven in 90% of cases, in its top three in 75%, and as the single most likely possibility in 56%. Four pre-specified safety criteria were established for supervisor intervention. Zero were triggered across the 98 patients who completed the study.

Where the AI lost ground was after the diagnostic step. Primary-care providers "outperformed AMIE in the practicality and cost effectiveness of Mx plans," the authors note, attributing that gap to the system's lack of access to electronic health records and its inability to conduct physical examinations.

Patient attitudes toward AI improved after the encounter and "remained at an elevated level after seeing the provider." Survey respondents described AMIE as "polite and effective at explaining medical conditions."