Copenhagen ML spots strokes on helpline calls better than humans

Call-takers at Copenhagen's medical helpline caught 52.7% of strokes coming through the line in 2021. A machine-learning pipeline, run retrospectively on the same calls, caught 63.0%.

That is the headline result of a study published in npj Digital Medicine by researchers at Copenhagen University Hospital, the dispatch-AI company Corti, and the Technical University of Denmark. The pipeline transcribes incoming calls via automatic speech recognition, then runs the transcript through an ensemble of five multi-layer perceptrons to classify each call as stroke or non-stroke. It was trained on 3,811 manually transcribed calls, totalling 173 hours of audio, drawn from the MH-1813 helpline and supplemented with 1-1-2 emergency-line data.

Positive predictive value was 24.9% for the model and 17.1% for the human call-takers (p < 0.0001). The authors note the operating point is adjustable: "The framework can be tuned to a sensitivity of around 73%, while still having a higher positive predictive value than the human call-taker."

Not everything comes out clean. The model performed significantly better in patients 65 and older than in those aged 18 to 64, and both the model and the call-takers produced higher PPV on men than on women. Stroke prevalence on the MH-1813 line in 2021 was 0.250% of all calls, so both human and machine are hunting a rare event in a lot of noise.

The authors frame the system as decision support rather than replacement: "The model should be used as a supportive tool for call-takers in the decision-making process, contributing to higher stroke recognition."