ML pipeline beats Copenhagen call-takers on stroke recognition

A machine learning pipeline recognised stroke with a sensitivity of 63.0% in a held-out 2021 test set of Copenhagen medical helpline calls, compared with 52.7% for the human call-takers who handled the same calls, according to a retrospective study published on nature.com. Positive predictive value rose from 17.1% to 24.9%, and the difference was significant at p < 0.0001.

The system has two stages: an automatic speech recognition model transcribes each call, then a text classification model labels it stroke or non-stroke. The authors trained it on calls from 1 January 2015 to 31 December 2020 in Copenhagen and tested on calls from 2021.

"Advanced stroke treatment is time-dependent and, therefore, relies on recognition by call-takers at prehospital telehealth services to ensure fast hospitalisation," the authors write. A dispatcher sensitivity of 52.7% means nearly half of stroke calls go unflagged at the point of dispatch.

The paper does not claim the model is ready to replace human triage. The authors describe it as something that "may become a supportive tool for call-takers, aiding in early and accurate stroke recognition." Stroke prevalence in the 2021 call set was 0.250%, so the improved 24.9% PPV still leaves roughly three in four flagged calls as non-strokes.