paper web signal

Socratic Skill Learning cuts agent-teaching tokens 61.7%

TL;DR

  • Issue-Guided SSL reached 0.805 transfer utility while cutting Educator tokens 61.7% versus the simpler Fixed-Window variant.
  • SkillTransferBench spans 35 task families and 408 executable cases, with primary evaluation on 22 families and 155 cases.
  • Fixed-Window SSL scored 0.801 against a 0.723 no-skills baseline, topping adapted Trace2Skill (0.761) and SKILL-KD (0.751).

The headline number from Socratic Skill Learning, a new preprint on arXiv, is a 61.7% cut in Educator tokens while transfer utility rose to 0.805. The Fixed-Window variant hit 0.801 against a no-skills baseline of 0.723. Two adapted baselines, Trace2Skill and SKILL-KD, came in at 0.761 and 0.751.

The paper describes its loop as "interleaving expert teaching with Student practice," with the educator guiding investigation while the student operates the environment and authors skills on its own. The authors also introduce SkillTransferBench, a benchmark of 35 task families and 408 executable cases; primary results run on 22 families and 155 cases.

Total acquisition tokens, counting both sides of the loop, dropped only 16.9% versus Fixed-Window. The big 61.7% Educator saving is partly reallocated to the Student side. The benchmark is the authors' own, and no external replication has landed yet.