ActionOPD Lifts Qwen3 Agents 3.3-6.9 Points on OOD Tool-Use

ActionOPD, an on-policy agent-distillation method posted to Hugging Face, trains a smaller language-model agent by rewarding it for taking actions that mean the same thing as a larger teacher's actions, rather than for reproducing the teacher's tokens. On four out-of-distribution tool-use benchmarks, Qwen3-4B and Qwen3-8B students trained against two black-box teachers posted gains of 3.3 to 6.9 percentage points over their base scores.

Token-level distillation, the paper notes, 'typically requires teacher token probabilities and compatible tokenization', a constraint that blocks distillation across model families. ActionOPD requires 'only generation access to a black-box teacher, not teacher token probabilities or tokenizer alignment', putting closed-API teachers in reach.

Training ran for 100 steps with 64 task starts per step, drawn from Agent World Model (AWM) and EnvScaler. Four student candidates were scored against three teacher samples per state, with a semantic matcher comparing tool names, canonicalized arguments and intent. Qwen3-4B and Qwen3-8B were paired with Qwen3.7-Flash and DeepSeek-V4-Flash in a 2×2 design. Qwen3-4B distilled from DeepSeek-V4-Flash moved the four-benchmark average from 30.8 to 37.7, a 6.9-point jump; Qwen3-8B paired with Qwen3.7-Flash posted the smallest gain at 3.3.

On τ²-bench specifically, the paper reports ActionOPD 'achieves competitive performance against offline distillation and token-level OPD.' A Self-ActionOPD variant, in which 'a same-model teacher receives privileged task information', is previewed as a preliminary exploration.

The gap between what open-weight students can learn from closed-API teachers has been a recurring thread in our fine-tuning coverage over the past three months.