Copenhagen study: ML outperforms call-takers on stroke flags

A machine-learning pipeline reading transcripts of Copenhagen 1-1-2 and 1813 helpline calls recognised stroke more often than the human call-takers who handled them, according to a study in npj Digital Medicine.

Using calls from 1 January 2015 to 31 December 2020 to train, and 2021 calls to test, the researchers report that call-takers reached a sensitivity of 52.7% with a positive predictive value of 17.1%. The model reached 63.0% sensitivity and a PPV of 24.9%, with p < 0.0001 across the differences.

The pipeline is two steps: an automatic speech recognition model transcribes the call, then a text classifier categorises it as stroke or non-stroke. Ground truth came from hospital stroke diagnosis within 5 days of symptom onset.

The authors keep the framing narrow. "The model should be used as a supportive tool for call-takers in the decision-making process," they write in the discussion. Stroke accounted for 0.250% of MH-1813 calls in 2021, so even the better classifier is working against a very thin base rate.