CodeMidas Turns 3,185 Codebases Into 5,545 Agentic RL Tasks

CodeMidas, an agentic pipeline described in a new paper on arXiv, turns implemented functionality in existing codebases into executable reinforcement-learning environments using source code as its only task-specific input. The resulting dataset carries 5,545 training tasks drawn from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains.

The angle is what CodeMidas skips. Existing methods, the authors write, "typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted." CodeMidas instead "allocates agentic compute to every stage of environment construction": agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and "validate and filter candidate tasks through execution checks and repeated solution rollouts."

Training MiMo-V2.5 on those tasks with GRPO "improves performance on all five diverse benchmarks," the paper reports, covering issue repair (DeepSWE +11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show "increasing the number of high-quality training tasks improves performance," and trajectory analysis credits the RL-trained agent with "increasing codebase exploration and more diverse self-verification."

The authors, including Bowen Ye, Lei Li and Fuli Luo, close directionally: "These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks." The abstract publishes no absolute baseline scores, no license or release plan for the dataset, and no accounting of the compute the agentic construction stage consumes.