Copenhagen ML tool outperforms call-takers at spotting stroke

In retrospective testing on 2021 calls to Copenhagen's medical helpline, a machine learning framework recognised stroke with 63.0% sensitivity, against 52.7% for the human call-takers who had handled the same calls. Positive predictive value rose from 17.1% to 24.9%. The study, published in npj Digital Medicine, reports p < 0.0001 on both metrics.

Stroke is rare in this stream. It accounted for only 0.250% of all medical helpline calls in 2021, which is part of why PPVs stay low even as sensitivity climbs. Training ran on 2015-2020 calls to medical helpline 1813 and the 1-1-2 emergency line, with 2,361,178 matched calls assembled against the Danish Stroke Registry. The speech-recognition stage was trained on 3,811 manually transcribed calls, totalling 173 hours.

The authors position the system as decision support, not replacement. "The model should be used as a supportive tool for call-takers in the decision-making process," they write.

Performance is not uniform. The paper reports significantly higher PPV and F1 scores in men than women, and better recognition in patients aged 65 and over than in those 18-64, for both the model and the humans it was benchmarked against. The authors also note they had "no data available to estimate potential bias related to race, ethnicity, language, accent, or dialects." Two of the co-authors are affiliated with Corti, the Copenhagen company that sells AI tools to emergency dispatch.