Lafa flow language model hits 2.54x AR throughput at 7B scale
TL;DR
- Lafa 7B reports 2.54x the throughput of its autoregressive SFT baseline at comparable accuracy on an H100, using self-speculative decoding with block size 16.
- The 8B configuration scores 63.1% macro-average across IFEval, BBH, MMLU-Pro, GPQA, GSM8K and HumanEval, versus 56.8% for the matched AR SFT baseline.
- Training uses fewer than 30B tokens across 0.5B, 7B and 8B scales, which the paper frames as 18x more efficient than Dream, LLaDA and Efficient-DLM.
Lafa, a continuous-flow language model scaled to 8B parameters, reports 2.54x the throughput of its autoregressive baseline at comparable accuracy on an H100 and trains on fewer than 30B tokens, according to the paper posted on Hugging Face.
The architecture pairs causal attention on the prefix with bidirectional, asynchronous flow inside each block, adapted from pretrained Qwen2.5 Instruct (0.5B, 7B) and Qwen3 8B Base checkpoints. The 8B configuration scores 63.1% macro-average across IFEval, BBH, MMLU-Pro, GPQA, GSM8K and HumanEval, versus 56.8% for a matched AR SFT baseline, with Efficient-DLM reported at 63.5% on the same suite. At 7B with block size 16, self-speculative decoding delivers "2.54× throughput of AR models at comparable quality"; dropping to block size 4 posts 2.0x throughput with a 2.5-point accuracy gain over AR SFT.
The data-efficiency claim is specific: fewer than 30B tokens across all three scales, which the paper frames as "18× more efficient than Dream, LLaDA, and Efficient-DLM" at comparable scales, against diffusion baselines said to use up to 500B tokens. Code and weights ship on GitHub and Hugging Face, slotting into a busy stretch of open-source model releases this week.
Originally reported by huggingface.co
Read the original article →Original headline: Large Flow Language Models (Lafa) Hit 2.54x AR Throughput at 8B With 30B Tokens of Training