Copenhagen ML model outperforms call-takers on stroke calls

Call-takers at Copenhagen's medical helpline recognized stroke on 52.7% of actual stroke calls. A machine-learning pipeline, trained on calls from 1 January 2015 to 31 December 2020 and tested on 2021 calls, hit 63.0% on the same held-out year. The study, published in npj Digital Medicine, reports p < 0.0001 for the difference.

The pipeline runs in two steps. An automatic speech recognition model transcribes each call, then an ensemble of five multi-layer perceptrons classifies the text as stroke or non-stroke. The authors, affiliated with Copenhagen Emergency Medical Services, Copenhagen University Hospital, the Technical University of Denmark and Corti, frame the system as decision support, not a replacement. "The model should be used as a supportive tool for call-takers in the decision-making process," they write, "contributing to higher stroke recognition and potentially boosting confidence in their decisions."

The positive predictive value tells a more sobering story. Call-takers flagged stroke correctly 17.1% of the time; the model, 24.9%. Three in four of its stroke alerts, in other words, still are not strokes. Stroke prevalence in the helpline's 2021 call volume was 0.250%, so even a better classifier operates under long odds.

The design is retrospective. The paper reports no live deployment and no measured effect on hospitalization or patient outcome.