ACLArena Diagnoses Non-Monotonic Forgetting in Agent Post-Training
TL;DR
- Training Qwen3-8B sequentially through math, search, e-commerce and instruction-following swings AIME26 accuracy from 25.83 down to 6.04, then back up to 10.21.
- The proposed Mixture of Low-Rank Experts pairs a frozen SDFT backbone with per-stage LoRA experts trained via reinforcement learning and routed by environment tag.
- MLE lands within a point of independently trained oracles: 49.7 vs 49.9 on NQ, 57.7 vs 56.0 on single-hop search, 32.9 vs 33.2 on τ³-Retail.
Training a Qwen3-8B agent through math, then search, then e-commerce, then instruction-following does not simply add capabilities. Its math score whipsaws: AIME26 climbs to 25.83 after the first stage, holds at 23.33 through search, collapses to 6.04 after the e-commerce stage, then rebounds partway to 10.21 by the end.
The ACLArena paper on Hugging Face calls this 'Agent Continual Learning' and reports that 'Sequential training exhibits non-monotonic capability evolution.' Later stages both disrupt and partially recover earlier skills, and the authors trace the pattern to which tokens the optimizer rewrites: 'Low-entropy tokens: ~95% remain stable across stages,' while high-entropy decision points get substantially rewritten.
Their proposed fix, a Mixture of Low-Rank Experts, builds a shared backbone via Self-distilled Fine-tuning, freezes it, then trains a per-stage LoRA expert with reinforcement learning. At inference time an environment tag picks which expert to compose onto the backbone.
Reported numbers put MLE close to independently trained oracles across all four domains: 49.7 on NQ against a 49.9 expert, 57.7 on single-hop search against 56.0, 32.9 on τ³-Retail against 33.2, and 85.0 on IF-Eval. On AIME26 it lands at 21.04, essentially matching the paper's Multi-teacher Mixed On-policy Distillation baseline at 21.25.
The runs use the Slime RL framework on 2× NVIDIA H200 nodes. The paper's limitations section is candid that experiments are 'limited to Qwen3-8B-Base and fixed 4-stage curriculum,' with generalization across model families unclear. It joins a busy stretch of continual-learning and post-training work on our fine-tuning tracker.
Originally reported by huggingface.co
Read the original article →Original headline: ACLArena Paper Diagnoses Non-Monotonic Forgetting in Multi-Stage Agent Post-Training