Copenhagen ML caught 63% of stroke calls, beating call-takers

A machine learning system trained on calls to Copenhagen's non-emergency medical helpline recognised 63.0% of stroke cases in a held-out year, against 52.7% recognised by the human call-takers working the same line. The gap held across every subgroup the authors tested in the paper, with p<0.0001.

The model was trained on calls placed between 2015 and 2020 to Copenhagen's medical helpline 1813 and emergency line 1-1-2, then tested on 2021 calls. The pipeline converts audio to log-Mel spectrograms, runs them through a convolutional and bidirectional LSTM stack for speech recognition, then feeds a bag-of-words vector into an ensemble of five multi-layer perceptrons. Stroke prevalence in the 2021 test set was 0.250% of the helpline's calls.

Positive predictive value told the same story: 24.9% for the model, 17.1% for the call-takers. Both numbers are low in absolute terms, with roughly three in four model flags not being strokes, but the authors frame the system as augmentation rather than replacement. 'Machine learning framework for recognising stroke in prehospital medical helpline calls may become a supportive tool for call-takers, aiding in early and accurate stroke recognition,' they write.

The words the model learned to attend to included 'left,' 'right,' and 'side,' the anatomical language that stroke descriptions turn on. Performance varied across demographic subgroups, with the authors noting greater gender-based disparities existed. The study is retrospective and single-city, and whether the Danish-language model carries over to other helplines and languages is not something the paper tests.