CMU Ships Synthetic Hospital, a 1,268-Patient EHR Benchmark Physicians Distinguish From Real Only 53% of the Time

CMU researchers published Synthetic Hospital, an open EHR benchmark of 1,268 synthetic patients and 5,602 encounters grounded in ICD-10-CM, SNOMED CT and LOINC. In a blinded review, physicians distinguished the synthetic charts from real patient charts only 53% of the time. Across 10 frontier and open models, the best reconstructed a longitudinal problem list at 0.73 severity-weighted F1—matching mean physician performance but well below the top human at 0.89.