TU Dresden on-premise AI nears cloud accuracy in Nature paper

A fully on-premise clinical AI agent from TU Dresden hit 90.0% diagnostic accuracy on a seven-disease MIMIC-IV task, within a point of the 90.7% scored by a GPT-5.2 cloud baseline on the same benchmark. The system, published in Nature Medicine on September 15, is designed to run locally on hospital hardware. The paper tested it on NVIDIA H200 and RTX PRO 6000 GPUs using open-weight models including Qwen-3.5, GLM-4.5-Air, and GPT-OSS.

Beyond the headline accuracy, the reliability framework is the interesting part. The team found that behavioral consistency across five runs discriminated correct from incorrect diagnoses better than internal likelihood scores or linguistic-certainty cues, with an AUC of 0.860. Used as a gate, filtering to cases where the model agreed with itself at a threshold of 0.90, the agent retained 49.4% of cases at 98.9% diagnostic accuracy, with only three residual errors in the kept subset. In a physician-review substudy of 181 cases, the automated evaluation and the clinicians' consensus agreed in more than 90 percent of cases, according to a Dresden press release.

Senior author Prof. Jakob N. Kather, Professor of Clinical Artificial Intelligence at TU Dresden, framed the design goal as "selective autonomy. These systems should support clinicians in decision-making, but never take over completely," adding that "responsibility for diagnosis and treatment will always remain with humans."

The evaluation has limits. Both main benchmarks derive from MIMIC-IV and represent one data environment; the five-run consistency estimation increased token usage roughly five-fold over single-pass inference, so the reliability gate costs about five times the local GPU compute per case. The team says the next step is testing under realistic clinical conditions.