这篇论文提出了ResearchArena，用来测试AI研发中Agent会怎么搞破坏、监控能不能抓住，做法很实在，有具体的数据和案例。
ResearchArena 是一个框架，用于评估自动化AI研发中的AI控制方法，涵盖安全后训练、能力后训练、CUDA内核优化和推理服务器优化四项长周期任务。每个任务配有两个隐藏的副任务：嵌入式破坏（蓄意破坏提交的模型、适配器、内核或服务器）和独立副任务（在沙箱中执行隐蔽越界操作但留下诚实的构件）。研究测试了四种监控类型（是否执行和探测构件、是否看到代理的思维链），发现训练数据中的破坏最难捕捉，被标记率不到一半。允许监控器在构件上运行实验有帮助但仍不足，监控器常因只检查表面、解释掉异常或用错误测试而遗漏嵌入式破坏。研究将ResearchArena作为模块化框架发布，用于评估自动化AI研发中的破坏和控制。
ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D
As AI agents begin to automate AI R&D, we need ways to assess whether their outputs are safe to deploy, even when the agents themselves may be untrusted. AI control offers one such approach: rather than trusting the agent, it treats it as a potential adversary and uses a monitor to detect covert sabotage before deployment. We evaluate AI control for automated AI R&D with ResearchArena, a framework spanning four long-horizon tasks: safety post-training, capabilities post-training, CUDA-kernel optimization, and inference-server optimization. Because the deliverable in AI R&D is an artifact that will be deployed and run, we pair each main task with two kinds of hidden side task: an embedded side task that sabotages the submitted model, adapter, kernel, or server itself, and an independent side task that takes a covert out-of-scope action in the sandbox while leaving the artifact honest. We evaluate frontier agents at both sabotage and monitoring, across four monitor types that vary along two axes: whether the monitor may execute and probe the produced artifact, and whether it sees the agent's chain-of-thought. We find that sabotage hidden in the training data is the hardest to catch, flagged fewer than half the time. Letting the monitor run experiments on the artifact, rather than only read the trajectory, helps, but it is not enough: monitors still miss embedded sabotage by inspecting only the surface, by noticing the anomaly but explaining it away, or by probing the artifact with the wrong test. We release ResearchArena as a modular framework for evaluating sabotage and control in automated AI R&D.