LILIE cuts text-to-image gFD 14 points vs pixel baseline

The headline number: training a text-to-image model with a new language-informed loss cuts gFD by 14 points, from 66.84 to 53.01, against a pixel-only baseline on the GPIC dataset. The method, named LILIE, comes from Suchir Agarwal, Keshigeyan Chandrasegaran, Ziyu Chen, Kyle Sargent, Juan Carlos Niebles, Li Fei-Fei and Jiajun Wu in a paper posted to arXiv on October 16.

LILIE, pronounced 'lily', is pitched as a 'Language-Informed and Lossless Image Encoder.' The authors describe it as 'an invertible image transform trained such that a vision-language model can recover captions from noised features.' The encoder itself is a normalizing flow transformer, trained jointly with a vision-language model via next-token prediction. That encoder then supplies a loss signal for text-to-image training.

The framing is a direct challenge to the pixel-space-loss line of work that has gotten end-to-end image generation to competitive quality without learned tokenizers. The authors ask whether 'language-informed supervision can provide a richer training signal than pixel loss for visual generation.' Their answer, for JiT models on GPIC, is yes by 14 gFD points. Two researchers we track shared the preprint the day it went up.

The abstract reports one dataset, one baseline, and one metric. It names no model sizes, no training compute, no ablations. The authors say code, encoders, and generative models are available at suchiragarwal.com/lilie.