Copenhagen helpline ML beats humans at spotting strokes

A machine learning system listening to Danish medical helpline calls flagged strokes with 63.0% sensitivity, against 52.7% for the humans taking the calls, according to a retrospective study in npj Digital Medicine.

The dataset is six years of calls to Copenhagen Emergency Medical Services, covering the Capital Region of Denmark's 1.9 million residents. The authors matched 2,361,178 audio files to records from a pool of 2,730,199 contacts, trained on 2015 to 2020 calls, and tested on 2021 calls. Stroke was the reason for 0.250% of them.

Positive predictive value tracked the same gap: 24.9% for the model versus 17.1% for call-takers, with p below 0.0001. The authors say the threshold can be retuned to reach roughly 73% sensitivity while still outperforming humans on PPV. The pipeline is plain: automatic speech recognition to transcripts, then an ensemble of five multi-layer perceptrons doing text classification.

The gains were not uniform. Performance gaps between model and call-takers were larger for women and for patients aged 18 to 64.

The authors are explicit about the ceiling. The system "should be used as a supportive tool for call-takers in the decision-making process, contributing to a higher recognition of patients with stroke," they write, and "cannot replace medical call-takers," because "the effectiveness of the model is fully reliant on the conversation between the call-taker and caller and the call-taker's ability to skillfully triage the patient." Commercial rights to the model sit with Corti, a Copenhagen startup whose co-founders are co-authors on the paper, which calls for a randomized controlled trial before implementation.