nature.com web signal

ML model outperforms call-takers on Copenhagen stroke calls

TL;DR

  • On 2021 test calls, the model hit 63.0% sensitivity and 24.9% PPV versus 52.7% and 17.1% for human call-takers (p < 0.0001).
  • Training ran on calls to Denmark's medical helpline 1813 from 1 January 2015 to 31 December 2020, with ground truth from the Danish Stroke Registry.
  • The authors position the framework as a supportive decision-support tool for call-takers rather than a replacement.

A machine learning pipeline tested on calls to Denmark's medical helpline 1813 recognised stroke more reliably than the call-takers handling the same calls, a retrospective study in npj Digital Medicine reports. On the 2021 test set, the model reached a "sensitivity of 63.0% (62.0–64.1%)" and "PPV of 24.9% (24.3–25.5%)"; the call-takers on the same calls managed "sensitivity of 52.7% (95% confidence interval 49.2–56.4%)" and "positive predictive value (PPV) of 17.1% (15.5–18.6%)". The authors report: "The machine learning framework performs significantly better (p < 0.0001)."

The system was trained on calls to the 1813 helpline from "1 January 2015 to 31 December 2020", with 2021 calls held out as the test set and ground truth taken from the Danish Stroke Registry. It pairs automatic speech recognition with a text-classification model running over the generated transcripts.

The authors frame the finding carefully: "The framework can assist human call-takers during medical helpline calls. Ideally, this would enable a higher recognition of patients with stroke." A PPV in the mid-twenties also means the model still mislabels most of the calls it flags, and stroke is a small share of the 1813 traffic it is sifting through.

The paper is blunt about what it could not test. The team did not publish a word error rate for the speech recognizer, and the Danish data gave them no way to measure bias by race, ethnicity, language or accent.

Shared on Bluesky by 1 AI expert