ML model outperforms call-takers on Copenhagen stroke calls

A machine learning pipeline trained on Copenhagen's non-emergency medical helpline recognised stroke calls with 63.0% sensitivity and 24.9% positive predictive value on 2021 calls, against 52.7% and 17.1% for the human call-takers handling the same calls, according to a retrospective study in npj Digital Medicine.

The team trained an automatic speech recognition and text classification framework on 1,391,301 calls to Denmark's medical helpline 1813 from 2015 to 2020, then tested it on 344,030 calls from 2021, cross-referenced against the Danish Stroke Registry. The improvement held at p<0.0001 across every metric, and the authors write that the framework "can thus be tuned to a sensitivity of around 73%, while still having a higher positive predictive value than the human call-taker".

"Our results showed that a machine learning framework could substantially improve stroke recognition in medical helpline calls compared to solely relying on human call-takers," the authors write, positioning the system as an aid rather than a replacement. The paper is explicit on that line: "This study does not imply that a machine-learning model can replace medical call-takers...Instead, the model should be used as a supportive tool."

Both the model and the call-takers posted higher PPV and F1 scores for men than for women at p<0.0001. The model additionally performed better in patients 65 and over than in the 18-64 group, at the same significance level.