Copenhagen ML tool outperforms call-takers on stroke triage

Human call-takers on Copenhagen's medical helpline recognized stroke in 52.7% of calls where the patient was later diagnosed with one. A machine-learning framework trained on the same system's call recordings hit 63.0%, and positive predictive value rose from 17.1% to 24.9%, a gap the authors report as statistically significant at p<0.0001.

The retrospective study, published in npj Digital Medicine, matched 2,361,178 of 2,730,199 contacts to the Copenhagen Emergency Medical Services archive and the Danish Stroke Registry. Calls from 2015 through 2020 trained the model; 2021 calls formed the held-out test set. The pipeline runs speech recognition first, using a convolutional network with bidirectional LSTM layers trained on 3,811 manually transcribed emergency calls, then feeds the transcript into an ensemble of bag-of-words classifiers.

The authors stop short of a replacement claim. "This study does not imply that a machine-learning model can replace medical call-takers," they write; "the model should be used as a supportive tool." They also report that the framework "can thus be tuned to a sensitivity of around 73%, while still having a higher positive predictive value than the human call-taker."

Performance is not even across groups. The framework scored higher PPV and F1 on male callers than female, and higher sensitivity, PPV, and F1 on callers 65 and over than those 18 to 64, at p<0.0001 for each. Stroke prevalence on the MH-1813 non-emergency line in 2021 was 0.250%.