SkillEvo Beats Self-Reflection by 23 Points on Agent Skills

An agent-skill evolution system called SkillEvo outperforms a self-reflection baseline by 23.0 points and a single-turn QA baseline by 15.4 points, according to a preprint posted to arXiv by Qianxi Yan and five co-authors.

The premise: today's agent skills are either hand-authored or produced in one LLM generation pass, with no closed loop through which they can improve from the interaction failures they actually cause. Prior work that does close the loop uses single-turn question-answering as the evaluator. That, the paper argues, is where evolution stalls. "Once the first round has patched the gaps that a single exchange can reveal, the evolution gradient decays," the authors write, and defects that only surface across multiple turns stay invisible.

SkillEvo has two parts. It repurposes multi-turn user simulation from an evaluation endpoint into a feedback generator, so that "follow-up questions expose defects layer by layer, so that every round of revision both consumes feedback and produces new feedback." It also replaces the scalar reject-gate common in these pipelines with what the authors call an "independent governance layer" that "actively repairs factual degradation and structural bloat."

The evaluation is small: six categories of cloud services, nine production skills, and ninety-eight skill-reference files. The abstract publishes only the two point-gains. It does not include baseline pass rates, per-category breakdowns, or cost figures for the simulator or the governance layer.