PapersSelected

Google Pro² Agent 在 MLE-Bench 拿下 76.0% 奖牌率,创最高纪录

Why we picked it

Google 把 Agent 卡住的原因写成了清单,还配了实验来验证每个修复方案,这才有了 MLE-Bench 76.0% 的最高成绩。做 Agent 工作流的人可以直接抄这个 Problem State 设计,写代码改 bug 的团队也能参考它的研究笔记思路。

Google 团队发布的 Pro² Agent 在完整 MLE-Bench 上取得 76.0% 的奖牌率,是该基准迄今最高成绩。在标准的 24 小时时限下,它与此前最好的 70.7% 持平;将时限延长到 48 小时后达到 76.0%。

Body

现有自主研究 Agent 的常见做法是反复修改代码直到指标变化,这容易让它们卡在错误区域且不自知。Pro² Agent 引入 Problem State:维护一份结构化清单,记录阻碍成功的具体问题及对应的候选修复方案。Agent 通过受控实验更新每个问题的权重和每个修复方案的效果,并生成可供人类阅读和复用的研究笔记。

MLE-Bench 是衡量 Agent 自主完成机器学习工程任务的常用基准,Pro² Agent 的成绩为后续 Agent 设计提供了明确参照。

Original · DAIR.AI

Brilliant work from Google and colleagues.

(bookmark it)

Their research agent reaches a 76.0% medal rate on the complete MLE-Bench, the highest reported on the full benchmark.

Autonomous research agents usually keep editing the codebase until a metric moves. On hard tasks, this often leaves them stuck in the wrong part of the solution space, because they never write down what they think is wrong.

Pro² Agent addresses this with a Problem State. It is a structured list of the problems that block success, and the candidate fixes for each one.

The agent runs controlled tests that update how important each problem is and how much each fix helps. With the standard 24 hours, it matches the best reported 70.7%. With 48 hours, it reaches 76.0%.

Each task also produces research notes that humans can review and reuse.

https://t.co/Bqax5RuBhA

Chat with Paper: https://t.co/I6I3NpumqH

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Pro² Agent, Google, MLE-Bench, agents.

Source attribution: curated from DAIR.AI (@dair_ai).