MultiAgent-OPD co-trains hierarchical agent teams, hits 38.68%
TL;DR
- The system reaches 38.68% macro avg@4 across seven datasets, beating Prompted Critic by 4.06 points and C-MAS + RL by 2.61 points.
- A frozen Teacher scores student-generated search and summary tokens, giving Searchers local process supervision beyond final-answer feedback.
- The shared-parameter Leader-Searcher-Answerer team is trained end to end by combining on-policy distillation with reinforcement learning.
MultiAgent-OPD, a new training recipe for hierarchical agent teams, reaches 38.68% macro avg@4 across seven datasets by co-training a shared-parameter Leader-Searcher-Answerer team end to end. The method combines reinforcement learning with on-policy distillation: a Teacher is first trained in a Searcher-Answerer workflow using answer correctness, then frozen and used to grade student outputs during team training.
The preprint reports that "the frozen Teacher scores student-generated search and summary tokens to provide local process supervision for the Searchers," while "the Leader learns from team success and the Answerer from candidate correctness." The design targets the thin feedback subagents normally receive when only the final answer is graded.
Gains over prior baselines are incremental: 4.06 percentage points above Prompted Critic, 2.61 above C-MAS + RL. "The team also solves more questions within four attempts," the authors write. Two researchers in our tracker circulated the paper. The abstract does not identify the seven evaluation datasets, name a base model, or disclose compute.
Shared on Bluesky by 2 AI experts
Originally reported by arxiv.org
Read the original article →Original headline: MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation