nature.com web signal

MedGemma: Google's open medical VLM tops MedQA at 86.2%

TL;DR

  • MedGemma, Google's open medical vision-language family built on Gemma 3, scored 86.2 on MedQA, which the paper calls the best open-model performance at release.
  • Clinicians rated 81% of MedGemma's chest X-ray reports as likely to produce the same or superior clinical decisions versus the originals.
  • The authors report a 500-fold computational cost difference between MedGemma 4B and the most expensive comparator model.

Google released MedGemma, an open family of medical vision-language models built on Gemma 3, in a Nature Medicine paper. The headline score: 86.2 on MedQA, which the authors say "represented the best performance for an open model" at release.

The collection ships 4B and 27B multimodal variants plus a 27B text-only model, alongside a companion vision encoder called MedSigLIP, "a medically tuned vision encoder derived from SigLIP." Against the base Gemma 3 models, the paper reports out-of-distribution gains of "2.6–10% in medical image question answering, 15.5–18.1% in chest X-ray finding classification and 10.8% in agentic evaluations."

Clinicians rated 81% of MedGemma's generated chest X-ray reports as "likely to result in the same or superior clinical decisions" compared with the originals. A fine-tuned variant hit a RadGraph F1 score of 30.3 on MIMIC-CXR. Inaccurate responses on open-ended clinical questions fell from 18.5% for Gemma 3 4B to 15.4% for MedGemma 4B, a 17% relative decrease. Two researchers we follow shared the paper on the day it posted.

The paper flags one blunt argument for the small variant: "a 500-fold difference in computational cost between MedGemma 4B and the most expensive comparator model." The results compare MedGemma against generative models of similar size; the authors frame it as a foundation for fine-tuning rather than a finished clinical product.

Shared on Bluesky by 2 AI experts