微软 AutoEnvScaling：模型自建 RL 环境来训练自己
New impressive work from Microsoft and collaborators. And it's one of my favorite recent topics: RL...
RL 训练缺环境，微软让 agent 在终端里自己造，Qwen3.6 从 41.1 涨到 53.3。
微软与合作者发布 AutoEnvScaling，由一个运行在终端里的 proposer agent 阅读被训练模型的 rollout 并据此设计新的 RL 环境。用自建环境训练后，Qwen3.6-35B-A3B 在 Terminal-Bench 2.1 上从 41.1 提升到 53.3。在终端中构建环境让有效环境占比比纯 prompt 生成最高提升 25 倍，每个有效环境的成本最高降低 75%。改用 GPT-5.6-sol 或 Claude Opus 5 担任 proposer 时，Qwen3.5-35B-A3B 分别比同规模静态环境池高出 4.3 和 3.8 分；当 proposer 看不到求解反馈时，收益会明显缩小。
New impressive work from Microsoft and collaborators. And it's one of my favorite recent topics: RL...
New impressive work from Microsoft and collaborators. And it's one of my favorite recent topics: RL environments. They show that Qwen3.6-35B-A3B goes from 41.1 to 53.3 on Terminal-Bench 2.1 by training on environments it designed itself. AutoEnvScaling gives environment design to a proposer agent that works in a terminal, looks at the rollouts of the model being trained, and builds new RL environments from them. The trained model's next rollouts guide the next round. Building environments in a terminal raises the share of valid environments up to 25 times over prompt-only generation, at up to 75% lower cost per valid environment. With GPT-5.6-sol or Claude Opus 5 as the proposer, Qwen3.5-35B-A3B scores 4.3 and 3.8 points above matched static environment pools. The gain shrinks when the proposer cannot see solver feedback. The pipeline also produces environments for software engineering, computer use, and GPU kernel work. arxiv.org/abs/2610.14163 Chat with Paper: academy.dair.ai/papers/autoenv… 💬 11 🔄 6 ❤️ 27 👀 2240 📊 16 ⚡