Copenhagen ML model beats call-takers at phone stroke triage

Researchers at Copenhagen Emergency Medical Services trained a two-stage speech-and-text model on six years of medical helpline calls and tested it against human call-takers on 2021 traffic: the model caught 63.0% of stroke cases (95% CI 62.0–64.1%) with a positive predictive value of 24.9%, against 52.7% sensitivity and 17.1% PPV for the call-takers themselves, a gap the authors report as significant at p < 0.0001.

The pipeline, described in npj Digital Medicine, first transcribes Danish-language calls using a convolutional-plus-bidirectional-LSTM speech model, then classifies the transcript as stroke or non-stroke using an ensemble of five multi-layer perceptrons on bag-of-words features. Training used calls from 1 January 2015 to 31 December 2020; the 2021 holdout was linked back to the Danish Stroke Registry for ground truth. Stroke accounted for only 0.250% of all calls on the MH-1813 medical helpline in 2021, which is part of why PPV is low on both sides.

The authors are careful about scope. "This study does not imply that a machine-learning model can replace medical call-takers," they write. "The effectiveness of the model is fully reliant on the conversation between the call-taker and caller and the call-taker's ability to skillfully triage the patient." They frame the system as something that "may become a supportive tool for call-takers, aiding in early and accurate stroke recognition."

The limitations are stated plainly: the work is retrospective, the mapping from audio to electronic records was incomplete, the word error rate of the speech model was not formally evaluated, and potential bias with respect to race, ethnicity, language, accent and dialect was not assessed.