H Company Ships NeoMME, an Apache 2.0 Multimodal Encoder
TL;DR
- NeoMME uses one bidirectional Transformer for text tokens and raw image patches, trained from scratch with a masked discrete-diffusion objective.
- NeoMME-Retriever-260M scores 0.523 nDCG@10 on ViDoRe v3, versus 0.261 for ColModernVBERT and 0.207 for ColSmol-256M at similar sizes.
- On an NVIDIA L40S at 2048×2048 input, the 260M retriever runs at about 51 pages per second, roughly twice ColModernVBERT's 26.
H Company has released NeoMME, a family of 260M and 800M multilingual multimodal encoders, under an Apache 2.0 license. The design skips the usual assembly of a pretrained vision tower plus a causal language model. In the team's own words: "A single bidirectional Transformer processes both text tokens and raw image patches, and we train the entire model from scratch with a masked discrete-diffusion objective."
On the retrieval side, the numbers are the pitch. NeoMME-Retriever-260M scores 0.523 nDCG@10 on ViDoRe v3, against 0.261 for ColModernVBERT and 0.207 for ColSmol-256M at similar sizes. The 800M variant lands at 0.556, which the post says is "within 0.002 nDCG@10 of ColQwen2.5 while using about 14× fewer parameters." Vultron Flash at 850M still edges it, at 0.565.
Throughput on an NVIDIA L40S with 2048×2048 input comes in near 51 pages per second for the 260M model, roughly twice ColModernVBERT's 26. The retriever returns a dense head and a late-interaction head in the same forward pass. Hierarchical token pooling paired with asymmetric quantization then compresses each late-interaction page representation from about 1.5 MB down to 6 kB, "255× smaller," while retaining more than 95% of baseline nDCG@10 on the same benchmark.
The technical report is credited to Aurélien Lac and Tony Wu, with a broader H Company team in the byline. All checkpoints ship under Apache 2.0.
Originally reported by lab-post
Read the original article →Original headline: H Company Ships NeoMME, 260M/800M Single-Tower Multimodal Encoder, Apache 2.0