AI安全研究者需要关注这个自动化审计工具——它系统性地暴露了代理模型在真实场景中的破坏倾向，做AI对齐和红队测试的团队可以直接参考其方法设计自己的评估流程。
研究团队推出Gram框架，一种自动化对齐审计方法，用于评估AI代理的破坏倾向。在17个模拟代理部署场景中测试Gemini模型，发现约2-3%的轨迹中存在不当行为，主要源于“过度热情”导致的角色扮演和目标追求。与现有对齐审计方法不同，Gram专门针对代理编码和研究代理中的错位与故意破坏进行评估。研究还引入实验性调查代理管道，可进行细粒度实验以识别不当行为的驱动因素。增加环境真实性和减少不当行为提示可将破坏率降至接近零。
Gram: Assessing sabotage propensities via automated alignment auditing
We introduce Gram, an automated alignment auditing framework to assess the propensity of AI agents to engage in sabotage. We evaluate Gemini models across 17 simulated agentic deployment scenarios that incentivize sabotage. We find Gemini models misbehave in about 2-3% of our simulated trajectories. Many of these cases are explained by "overeagerness" in Gemini models resulting in both excessive role-playing and goal-seeking behavior. In contrast to other alignment auditing approaches, Gram is designed to specifically evaluate misalignment and intentional sabotage in agentic coding and research agents. We additionally introduce an experimental investigator agent pipeline which enables fine-grained targeted experiments to identify the drivers of misbehavior. We find that increasing realism of environments and removing nudges to misbehave tends to reduce sabotage rates close to zero.