PapersSelected

AutoRL:用智能体监督强化学习训练,Qwen3 AIME 成绩提升 20 个百分点

Why we picked it

把 RL 训练调参交给智能体团队:Perceptor 看曲线、Proposer 出方案、Orchestrator 执行热更新。Qwen3 数学基准涨 20 个百分点,连故意写坏的配方都能自动修。做训练管线或想省调参人力的人值得细看。

一篇新论文提出 AutoRL 框架,让 AI 智能体接管强化学习训练的实时监督,替代固定人工流程。在 Qwen3 上,AIME 2024 准确率从 46.67% 提升到 66.67%,AIME 2025 与 HMMT 2025 各提升 11.67 分。

Body

AutoRL 的架构分三层:Perceptor 智能体持续观察训练曲线、rollout 与评估结果并标记异常;三个 Proposer 智能体诊断问题并提出修改方案;Orchestrator 审批后通过热更新或从 checkpoint 重启来应用改动。在 agentic coding 场景中,配合 LongCat-Flash-Lite,pass@1 从 52.29% 升至 65.13%,且是在长时间无人值守、包含中断的运行中完成的。作者还故意给出一个有缺陷的训练配方,AutoRL 成功找到并修复了问题。

论文指出,智能体已能胜任常规的 RL 监督工作,但每次改动仍需大量算力来验证,这是当前的主要瓶颈。

Original · elvis

Self-improving AI is one of the most discussed ideas right now. This is one of the most exciting papers I've read recently on the topic. (bookmark it) It proposes AutoRL, which lifts Qwen3 on AIME 2024 from 46.67% to 66.67% by letting agents supervise a live RL run instead of following a fixed recipe. AutoRL works like an RL researcher on call. Perceptor agents watch training curves, rollouts, and evaluations and flag problems. Three Proposer agents diagnose the issue and suggest changes. An Orchestrator approves them and applies them with a hot update or a restart from a checkpoint. On math, AIME 2025 and HMMT 2025 each improve by 11.67 points. On agentic coding with LongCat-Flash-Lite, pass @1 rises from 52.29% to 65.13% over a long unattended run with interruptions. Given a deliberately broken recipe, AutoRL found and fixed the problems. The authors say agents can already handle routine RL supervision. Validating each change still takes a lot of compute. arxiv.org/abs/2611.17790 Chat with Paper: academy.dair.ai/papers/autorl-…📊 15 ⚡

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include AutoRL, Qwen3, reinforcement learning, agents.

Source attribution: curated from elvis (@omarsar0).