AutoEnvScaling: agents build their own RL training environments

A new preprint on arXiv proposes a training setup in which agents build the reinforcement-learning environments used to train other agents.

The paper is called AutoEnvScaling, and its central claim is numerical. A proposer agent that builds new RL environments from a solver's own rollouts can, per the abstract, 'increase the valid-environment rate by up to 25 times compared to prompt-only generation, at up to 75% lower cost per valid environment.' The problem it is designed around, in the authors' words: terminal-agent training needs 'diverse and verifiable environments, but these environments are still mostly designed by human researchers, so their supply grows only as fast as researchers can work.'

The headline numbers come from Terminal-Bench. Training Qwen3.5-35B-A3B on environments produced by GPT-5.6-sol 'improves its Terminal-Bench 2.1 score by 4.3 points over the matched Tmax baseline.' When Qwen3.6-35B-A3B plays both roles, generating environments and solving them, the authors report that it 'gains 12.2 points on Terminal-Bench 2.1 and raises its Terminal-Bench 4.0 score from 1.6 to 3.2.'

The authors frame the dual-role setup explicitly: by making one model do both environment generation and solving terminal tasks, 'it lets one model co-evolve in both roles.' They call this a 'core step toward RSI,' recursive self-improvement.

The abstract says the pipeline 'generalizes across agentic domains,' listing 'software engineering, computer use, professional work, and GPU kernel programming,' though no per-domain numbers appear in the abstract itself. It also reports only deltas on Terminal-Bench 2.1, not absolute scores, and lists no author affiliations in the material retrieved here.