SteerDistill Closes W4A16 Agent Gap to BF16 Within a Point

Four-bit quantized LLM agents, trained with a BF16 teacher correcting them in the loop, can land within one point of the full-precision baseline across five agentic benchmarks, according to SteerDistill, a paper by Genming Cui and coauthors posted this week.

Under W4A16, the method "exceeds the strongest quantized baseline by 1.67--2.18 points in average score" and "stays within one point of BF16" across three models (Qwen3-8B, Qwen3.5-4B and Gemma 4 12B-it) on BFCL V4, τ²-Bench, WebShop, ALFWorld and Search QA. On τ²-Bench specifically, it beats QeRL by 1.84 percentage points in success rate while using "67% fewer training GPU hours" and producing shorter successful trajectories than the quantized baselines.

The diagnostic behind the method carries the sharper number. The paper reports that in a BFCL V4 multi-turn setup, "the student's unguided retries yield a correct action at only 9.6% of divergent turns"; with a frozen teacher steering those retries and iteratively adjusting its guidance from student execution feedback, "teacher steering raises turn-level Pass@4 from 9.6% to 82.3%." The student is then trained without guidance to match the steered student at corrected turns and the teacher at other turns, so at inference time it runs alone.

The abstract leaves pieces open: it does not break out per-model or per-benchmark absolute scores, and the 67% training-hour saving is quoted on τ²-Bench against QeRL rather than averaged across the five tasks. The release lands in a visible run of quantization and agent-training work on our feed, including Huawei's MACaQ pipeline earlier today, and the authors have posted a project page and code alongside the preprint.