Paper: distilled skills match reasoning at 2.7-6x fewer tokens

Reasoning-mode language models spend 3 to 6 times more output tokens than their non-reasoning counterparts on multi-step agentic tasks, and much of that cost, the paper argues, goes to re-deriving procedures shared across every episode in the same domain.

The workaround is small. A coding agent reads a corpus of existing trajectories from a training split, then compiles a compact natural-language skill that gets injected into the non-reasoning model's system prompt. In a preprint posted to arxiv, Agamdeep Singh and co-authors report that across four agentic benchmarks (ALFWorld, tau²-bench telecom and retail, and SpreadsheetBench-Verified), skills "recover 55%-100%+ of the reasoning gap for GPT-5.4-mini on held-out tasks," beating reasoning mode outright on two of the four while "emitting 2.7-6x fewer output tokens and zero reasoning tokens."

Reasoning traces themselves are not a prerequisite. "Skills distilled from non-reasoning trajectories alone remain competitive with skills distilled from paired reasoning/non-reasoning corpora," the authors write, with "domain-dependent differences between the two sources."

The framing is a search one: "test-time reasoning is deep search inside a single episode, re-paid at every deployment, while corpus distillation is wide search across episodes, paid once." The residual gap on telecom and SpreadsheetBench, the authors note, is where "genuinely per-instance deep search remains necessary."