Microsoft's MAI-DxO Beats Doctors 83% to 36% on NEJM Cases

Microsoft's MAI-DxO, an orchestration layer that runs AI agents as a mock physician panel, correctly diagnosed 83% of complex case reports from the New England Journal of Medicine. The generalist doctors and internists benchmarked against it, working with reference materials and without a specialist team, got 36%. The paper was published Thursday in Science. TechCrunch reports that Microsoft tested the harness against frontier models from OpenAI, Google, Anthropic, Meta, and DeepSeek, with the strongest accuracy coming paired with OpenAI's GPT-5 and GPT-5.5.

Microsoft AI CEO Mustafa Suleyman is pitching the work as a step toward "medical superintelligence," which Microsoft defines as "the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care." Dominic King, VP of Health at Microsoft AI, put the use case more plainly: "The biggest challenge in healthcare delivery at the moment is the growing mismatch between demand and supply of medical expertise."

Most of the lift sits in the underlying model, not the harness. DxO moved Gemini 2.5 Flash from 46.4% to nearly 69%, pushed GPT-5.5 from 72% to nearly 87%, and on GPT-5 trimmed roughly $3,000 off an approximately $8,400 per-case cost while adding only about four accuracy points. TechCrunch notes that "the study's own data suggests that much of the improvement comes from more powerful underlying AI models." The NEJM cases being graded are also usually handled by specialists or physician teams, who were not part of the comparison.

Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston to evaluate DxO's clinical reasoning in hospital settings, and plans a healthcare foundation model with Mayo Clinic targeted for 2027. "I really believe that in 2027, we will be on the verge of a coding style breakthrough for medical records," Suleyman said. It joins a steady run of Microsoft AI stories we have tracked this quarter.