Copenhagen ML model spots stroke calls better than humans

A machine learning model flagged stroke in Copenhagen's non-emergency medical helpline calls more often than the human call-takers did, correctly identifying 63.0% of stroke cases against the humans' 52.7%, according to the retrospective study in npj Digital Medicine.

The work draws on calls to Copenhagen's medical helpline 1813, training on 2015 through 2020 and testing on 2021. 2,361,178 contacts were successfully matched to patient outcomes; stroke made up 0.250% of 2021 calls. At a calibrated threshold, the algorithm reached a "sensitivity of 63.0% (62.0–64.1%) and a PPV of 24.9% (24.3–25.5%)", versus a "sensitivity of 52.7% (95% confidence interval 49.2–56.4%) with a positive predictive value (PPV) of 17.1% (15.5–18.6%)" for the call-takers.

"Calls are first transcribed using an automatic speech recognition model and then categorised as stroke or non-stroke using a text classification model," the authors write. The classifier is an ensemble of multi-layer perceptrons on a bag-of-words representation of the transcript. Corti, an AI company, co-authored the paper with clinicians at Copenhagen University Hospital and Denmark's prehospital services.

The model "significantly outperformed the call-takers on all metrics for each sex," per the paper, with larger relative gains for women. The authors frame the system as assistance rather than replacement: "A machine learning framework for recognising stroke in prehospital medical helpline calls may become a supportive tool for call-takers, aiding in early and accurate stroke recognition." No live deployment is reported in the paper.