NPCBench finds top multimodal models hit just 23% on patient care

Among 21 multimodal models tested on NPCBench, a new nasopharyngeal carcinoma benchmark from The Hong Kong Polytechnic University, infix.ai, Sun Yat-sen University and Monash University, strict patient-level success tops out at 23%, reached by Claude-Sonnet-4.6. Gemini-3.1-Pro-Preview, the strongest at individual tasks, posts 89.9% care accuracy but just 11.0% strict CarePath success.

The gap is the entire point. "Strong performance on individual tasks does not ensure jointly correct diagnostic and care answers for the same patient," the authors write in the NPCBench paper, which bundles 4,777 component items with a 100-patient CarePath cohort covering 417 scored diagnostic and care tasks.

Evidence-grounded T-stage success sits below 5.0%. For Gemini-3.1 specifically, 72 of 100 patients have all care decisions correct despite at least one incorrect pathology or MRI T-stage label, meaning correct answers often rest on flawed anatomy. Consultation safety follows the pattern: GPT-5.1 leads that suite at 58.4 yet still trips major-error criteria in 14.6% of responses, one more signal flagged in our rolling healthcare AI coverage this quarter.