Copenhagen ML pipeline tops call-takers on stroke calls

A machine-learning pipeline trained on six years of Copenhagen medical helpline recordings recognised stroke calls with 63.0% sensitivity and a positive predictive value of 24.9%, outperforming the human call-takers who achieved 52.7% sensitivity and 17.1% PPV on the same 2021 test set (p < 0.0001).

Published in npj Digital Medicine, the system first transcribes calls using an automatic speech recognition model, then classifies the text as stroke or non-stroke. Training used calls from 1 January 2015 to 31 December 2020; 2021 calls from the MH-1813 medical helpline served as the held-out test. The authors describe the work as "using the largest collection of audio calls from patients with stroke to date."

Both the model and the call-takers "exhibited significantly higher PPV and F1 score in men than in women," though the paper adds that "the model exhibited less bias than call-takers did." The model also "performed significantly better in the 65+ group than in the 18–64 year group." The authors are explicit that this is not a replacement system: "the model should be used as a supportive tool for call-takers in the decision-making process."