FactEmbed Factorizes MLLM Embeddings for Composed Retrieval

Composed Image Retrieval asks a model to find a target image given a reference image and a modification text. A new preprint posted to Hugging Face Papers argues that current MLLM-based systems are still leaving compositional understanding on the table, because they map a composed query into a retrieval embedding "via contrastive supervision alone, without explicitly resolving the intended visual modifications."

The paper's answer is FactEmbed, which "decouples compositional query comprehension from building discriminative embeddings." Instead of pushing the composed query into a single vector, the model runs a set of factorized latents through an "intermediate target imagination objective" before shaping the retrieval representation, with a factor-level contrastive loss and a distillation into a deeper attention layer of the MLLM.

The reported numbers: on Refined-CIRR, R@1 of 76.29 against ReCALL's 73.59; on Refined-FashionIQ, R@10 of 87.21 against 84.81; on the CIRR test set, an average of 79.06. Swapping the Qwen2.5-VL-7B backbone for Qwen3-VL-8B lifts the CIRR average to 81.71.

Inference stays cheap: the [EOS] token carries a global embedding for the initial retrieve, and the mean-pooled factorized latents only rerank a short candidate list, at roughly 0.17ms per query. The authors flag that K=8 is fixed rather than query-adaptive, and that evaluation is limited to CIRR and FashionIQ.