论文

CodeMidas:以源代码为输入自动构建编码智能体 RL 训练环境

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

精选理由

把现成开源代码库自动变成智能体 RL 练习题,MiMo-V2.5 训完 ProgramBench 涨了 17%。

CodeMidas 是一条智能体流水线,只以源代码为任务输入,把开源代码库中已实现的功能自动转成可执行的 RL 训练环境。流水线让智能体分阶段工作:探索代码功能生成行为规范、基于原始代码的执行结果构建测试、再通过执行检查和多次求解 rollout 过滤候选任务。最终数据集包含来自 3,185 个开源代码库的 5,545 个训练任务,覆盖 23 种编程语言和 15 个技术领域。用 GRPO 在这些任务上训练 MiMo-V2.5 后,五个基准全部提升:ProgramBench +17%、DeepSWE +11.7%、Terminal-Bench v2.1 +8.5%。消融实验显示高质量任务数量越多性能越好,轨迹分析表明训练后的智能体更常探索代码库并进行更多样的自我验证。

原文 · arXiv cs.AI

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present CodeMidas, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks.