IDeaL distills students without real data, matches ImageNet run

Students distilled from multiple teacher networks can match, and sometimes surpass, students distilled from a same-sized slice of real ImageNet data, without ever touching real images. That is the claim in IDeaL, a paper by Feyza Yavuz, Mert Bülent Sarıyıldız and Diane Larlus accepted at ECCV 2026 and already picked up by two of the researchers we follow.

Multi-teacher distillation normally trains a student to mimic several teachers on the union of their training sets. The authors question that assumption, first probing how far a student gets when teachers see only noise, then designing IDeaL samples: synthetic inputs generated using 'decorrelation losses at both patch and image levels' and tailored to each teacher. The paper reports that 'given a limited budget of 1K images for distillation, students distilled using our IDeaL samples match or surpass the performance of those distilled using a 1K-image subset of ImageNet.'

The authors describe the results as 'surprisingly competitive' and say they 'substantially narrow the gap with students distilled from real images.' The abstract publishes no per-architecture numbers or teacher lineup, so the size of the remaining gap is left to the full paper.