Category-aware RL fix lifts SWE agents 5.39pp on Pro-618

Pooled reinforcement learning of software engineering agents can lift performance in some categories of repository task while degrading others, an effect a new arxiv preprint names the "category see-saw" and argues is masked by aggregate SWE-bench-style resolution rates. The authors' category-aware alternative, distilled into a single deployable model, hits 58.04% mean resolution on Pro-618 and 59.00% on SWE-bench Multilingual, up 5.39 and 2.78 percentage points on the base model.

"Gains in some categories coincide with regressions in others, while aggregate resolution obscures these changes," the abstract reports. The proposed fix trains one RL agent per category and then merges them. Same-origin category experts cycle through what the paper calls "Refresh-Repair-Expand (RRE): the updated policy refreshes instance mastery, reuses its own verified successful trajectories for Repair SFT, and reselects tasks for further RL." A label-routed multi-teacher on-policy distillation step, MOPD, consolidates the experts "into one deployable student," with a ReLU gate that keeps only each teacher's improving direction over the reference.

The recipe is self-contained. "Expert training and policy integration require no external model to provide solution trajectories or action targets," the authors write, which sets it apart from distillation pipelines that lean on a stronger closed model as teacher.

Beyond aggregate scores, the evaluation reports per-category resolution, the minimum category lift over each joint-RL baseline, and expert-gain recovery. The abstract names neither the base model that was improved by those 5.39 and 2.78 points nor the category taxonomy that the SWE Labeler produces, and it publishes no per-category resolution numbers.