Kaggle 联合 Eigent_AI 发布 AgenticRL-Bench 基准,GPT-6.1 Sol 以 22.8% 居首
做 RL 训练工程的朋友应该看看这个:Kaggle 和 Eigent 把 verl、SGLang 这些栈里真实的调试事故做成了基准,智能体要真修好才算过。最扎眼的是 7/22 的任务会触发各家安全分类器,实验室松紧不同会直接扭曲榜单成绩。选型或做内部评测的团队可以参考这套任务设计。
Kaggle 与 Eigent_AI 合作发布 AgenticRL-Bench,这是一个评测编程智能体能否诊断并修复真实 RL 训练事故的基准,覆盖 Miles、slime、verl、SGLang、Megatron、Harbor 等主流 agentic RL 技术栈。基准源自 Eigent 的 22 个后训练 RL 任务,Kaggle 排行榜对其中 19 个可验证任务打分,任务包括 Verl Upgrade Gate、Reward Determinism under Load、Trial Fault Attribution、SQL Partial Credit Exploits 等。
评测规则严格:任务只有在所有隐藏检查项全部通过时才算成功,每个智能体每题有三次尝试机会,由 Kaggle 在 Modal 上运行。当前榜单前三名为 GPT-6.1 Sol(22.8%)、Gemini 4 Carbon(19.3%),GPT-6 Astra 与 Claude Opus 5.5 并列第三(14.0%)。
复现过程中还发现一个反常现象:22 个任务中有 7 个会触发实验室的安全分类器,且不同实验室分类器的触发频率差异很大,这意味着此类能力基准的成绩在相当程度上反映的是实验室的风险容忍度,而不仅是模型真实水平。
Today, in collaboration with @Eigent_AI, we’re announcing AgenticRL-Bench, a benchmark that evaluates whether coding agents can diagnose and fix real incidents across the agentic RL stack (Miles, slime, verl, SGLang, Megatron, Harbor).
AgenticRL-Bench is built from 22 post-training RL tasks, and the Kaggle leaderboard scores agents on the 19 verifiable ones. These cover real agentic-RL engineering work, from debugging to training, in tasks such as Verl Upgrade Gate, Reward Determinism under Load, Trial Fault Attribution, and SQL Partial Credit Exploits. A task passes only if every hidden check passes, and each agent gets three attempts per task, run by Kaggle on Modal.
Reproducing AgenticRL-Bench surfaced an unusual pattern in capability benchmarking today: results often reflect frontier labs' risk tolerance as much as model intelligence. Seven of Eigent's 22 post-training RL tasks trip lab safety classifiers, a testament to the realism of the benchmark, and how often those classifiers trigger varies widely between labs, making it harder to measure true capability.
Current Leaderboard: 🥇 GPT-6.1 Sol — 22.8% 🥈 Gemini 4 Carbon — 19.3% 🥉 GPT-6 Astra and Claude Opus 5.5 — 14.0%
Check out the live leaderboard here: https://t.co/tDrm9ApWKE
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include AgenticRL-Bench, Eigent_AI, Kaggle, verl.
Source attribution: curated from Kaggle (@kaggle). Cross-source corroboration: 11 independent sources have covered this development (including lmarena.ai, Artificial Analysis, 宝玉), confirming industry-wide signal strength.