Lightning Weave lifts Qwen3.5-4B math to 64%, cuts tokens
TL;DR
- On Qwen3.5-4B, Lightning Weave lifts HMMT 2025 math accuracy from 59.2% to 64.0% while cutting response tokens by 10.7%.
- On LiveCodeBench v5, the same framework moves coding accuracy from 41.7% to 54.2% with 9.6% fewer response tokens.
- The setup scores cached trajectories once, avoiding the cost of running multiple live anchor models concurrently during student training.
Lightning Weave, a post-training framework from Yecheng Wu, Song Han and Han Cai, lifts Qwen3.5-4B's HMMT 2025 math accuracy from 59.2% to 64.0% while trimming response tokens by 10.7%, and on LiveCodeBench v5 pushes coding accuracy from 41.7% to 54.2% with 9.6% fewer tokens, per the abstract posted September 13.
The technique pulls capabilities out of independently trained specialist models and folds them into a single student via on-policy distillation. 'Each acquired capability is represented by the policy shift from the model before post-training to the resulting specialist,' the abstract states.
The efficiency argument extends to the training loop: the method scores cached trajectories once, eliminating the need to run multiple live anchor models concurrently during student training. Two researchers on our radar surfaced the paper the day it went up.
The numbers are self-reported on one 4B base, and the paper is a work in progress with code release still planned.
Shared on Bluesky by 2 AI experts
Originally reported by arxiv.org
Read the original article →Original headline: Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition