nature.com web signal

Georgia Tech maps 31 medical-AI cases past treating doctors

TL;DR

  • Georgia Tech's Gennie Mansi and Mark Riedl analyzed 31 US legal cases involving healthcare AI to map where patient harms occur.
  • The pattern points to insurers, state health departments and care facilities, not the treating physicians who currently anchor AI accountability frameworks.
  • The paper calls for distributed liability across stakeholders and AI tools designed to help patient advocates pursue legal recourse.

The medical AI cases stacking up in US courts mostly aren't about doctors. A Nature Health analysis of 31 legal cases in the USA finds patient harm running through insurance algorithms, state screening systems and care facilities, parts of the delivery chain the treating physician does not control.

Georgia Tech researchers Gennie Mansi and Mark Riedl argue that the medical, legal and technology communities "often centre physicians' role in preventing and mitigating harms from medical artificial intelligence (AI) tools," while the actual pattern in litigation points elsewhere. Patient care, they write, depends on "a complex web of stakeholders" that includes "physicians, state health departments, health insurers, care facilities, among others," and "many AI systems deployed across their healthcare delivery negatively impact their care."

Coverage of the paper puts the accountability gap plainly. "The law still frames accountability as if a single physician stands between a patient and every algorithmic decision," bioengineer.org notes. In practice, "a patient's lawyer must identify which entity's conduct was negligent, prove causation, and often overcome trade-secret protections," a burden that compounds when a decision has passed through vendor, insurer and agency systems before it reaches the exam room.

The case pool draws on three public trackers, DAIL, the O'Neill Institute litigation tracker and AIAAIC, and features systems including nH Predict, the Oregon DHS Safety at Screening tool and the Babylon Health Triage System. Mansi and Riedl close with two proposals: restructure liability so it reflects every stakeholder shaping how AI reaches patients, and design tools that let patient advocates and lawyers mount a case.

Shared on Bluesky by 1 AI expert