ML pipeline beats Copenhagen dispatchers at spotting stroke calls
TL;DR
- A machine learning pipeline recognised 63.0% of stroke calls in Copenhagen EMS recordings, against 52.7% for human call-takers (p < 0.0001).
- The model's positive predictive value reached 24.9% versus 17.1% for call-takers, using 2015-2020 calls for training and 2021 for testing.
- The authors say the framework can be tuned to around 73% sensitivity while still beating the human call-taker's positive predictive value.
Copenhagen's emergency call-takers recognised stroke on 52.7% of the calls where it had occurred. A speech-recognition-plus-text-classification pipeline, run on the same calls, recognised 63.0%.
The numbers come from a retrospective study in npj Digital Medicine using calls to Copenhagen Emergency Medical Services from 1 January 2015 to 31 December 2020 for training and 2021 calls for testing, with 2,361,178 calls matched to audio. Call-takers' positive predictive value, the share of their stroke suspicions that actually were stroke, was 17.1%. The model's was 24.9%. The difference reached p < 0.0001.
The authors keep the framing cautious. "The model should be used as a supportive tool for call-takers in the decision-making process, contributing to a higher recognition of patients with stroke," they write. They also note the pipeline is tunable: "The framework can thus be tuned to a sensitivity of around 73%, while still having a higher positive predictive value than the human call-taker."
One explanation offered for the human gap is exposure. "Certain stroke symptoms are so rare that some call-takers may never encounter them, increasing the risk of false negatives," the paper notes. Two of the first-authors list Corti as an affiliation alongside Danish universities. The abstract reports retrospective performance only; no live deployment or patient-outcome data appear.
Shared on Bluesky by 1 AI expert
Originally reported by nature.com
Read the original article →Original headline: A retrospective study on machine learning-assisted stroke recognition for medical helpline calls - npj Digital Medicine