Copenhagen ML tool outperforms dispatchers on stroke calls
TL;DR
- Framework reached 63.0% sensitivity and 24.9% PPV versus 52.7% and 17.1% for human call-takers (p<0.0001).
- Audited 2,361,178 Copenhagen emergency calls from 2015-2020, with a 2021 held-out test year cross-referenced against the Danish Stroke Registry.
- Model could be tuned to around 73% sensitivity while keeping PPV above call-takers, and showed less sex bias than human dispatchers.
A machine learning framework retrospectively tested on Copenhagen emergency calls flagged more strokes than human dispatchers, reaching 63.0% sensitivity against 52.7% for call-takers and lifting positive predictive value from 17.1% to 24.9% (p<0.0001). The paper, published in npj Digital Medicine, audits 2,361,178 calls matched to audio across 2015-2020 training data and a 2021 held-out year, cross-referenced against the Danish Stroke Registry.
The system is a two-stage pipeline. Automatic speech recognition transcribes the call, then an ensemble of five multi-layer perceptrons trained on bag-of-words features classifies it. Lead authors Jonathan Wenstrup, Jakob Drachmann Havtorn and Lasse Borgholt, working across Copenhagen University Hospital, Corti, Copenhagen Emergency Medical Services, the Technical University of Denmark and Aalborg University, write that the goal was "to develop and assess the potential of machine learning in improving prehospital stroke recognition during medical helpline calls."
Operating point is a knob, not a verdict. The paper reports the framework can be tuned to "improve sensitivity to approximately 73% while maintaining PPV above that of human call-takers," a trade that matters because stroke calls are rare: prevalence on the MH-1813 helpline sat at 0.250% of 2021 calls. An occlusion analysis showed the model had learned stroke-specific vocabulary and symptom descriptors, and the authors report it "exhibited less sex bias than call-takers," though both still performed better on male patients. Accuracy was stronger on patients 65 and older than on those aged 18 to 64.
This is retrospective audit work, not a prospective trial. There is no evidence yet that pushing these alerts to a live dispatcher changes door-to-needle times or patient outcomes, and the corpus is Danish-language calls routed through one city's EMS.
Shared on Bluesky by 1 AI expert
Originally reported by nature.com
Read the original article →Original headline: A retrospective study on machine learning-assisted stroke recognition for medical helpline calls - npj Digital Medicine