Found first: a primary source the press has not covered yet.
Researchers at NYU Langone Health analyzed 127,833 queries submitted to an institutional LLM assistant by 6,342 clinicians over eight months and found that documentation and administration accounted for 36.2% of real-world use. A paper submitted October 8, 2026 reports that the median clinical AI benchmark includes no documentation tasks at all and aligns with only 31% of the actual task mix from that deployment.
What the source says
Krithik Vishwanath and colleagues at NYU Langone Health assembled the Clinical AI Benchmark Atlas by mapping 58 public benchmarks against real clinical queries drawn from 6,342 healthcare providers across 35 specialties over an 8-month rollout. In the real deployment, documentation and administration made up 36.2% of queries, knowledge retrieval 28.9%, and diagnosis 3.7%. The median benchmark contained no documentation tasks and shared 31% of the real-world task mix. Across all real clinical queries, 36.1% (95% CI 34.8 to 37.5%) could not be answered well as posed: 23.4% required material assumptions to answer and 12.7% could not be answered at all. A separate review of 4,609 AI-in-medicine studies published through September 2025 found that 77.3% used no real clinical data of any kind.
Why it matters
Benchmark scores guide decisions about which clinical AI tools to deploy and which to avoid. If benchmarks concentrate on diagnosis while real clinical workloads concentrate on documentation, high scores on those benchmarks offer limited signal about actual deployment utility. The 4,609-study review finding situates the benchmark gap inside a wider pattern: the published AI-in-medicine literature has been built predominantly on evaluations that never touched real clinical data. Institutions selecting tools on the basis of published benchmark performance are working from a task distribution that does not match the work.