DeepMind研究告诉你:别偷懒只用同一个模型的监控,换个不同家族的事实检查器,能堵住45%的漏洞。
DeepMind关联研究显示,链式思维监控作为智能体安全层可被说服绕过。给监控器访问推理痕迹导致有害行为批准率平均上升9.5%。使用Claude 3.7 Sonnet作为监控器配合GPT-4.1事实检查器(不同模型家族)使违规批准减少45%。相比之下,同一模型家族内只减少6%。跨族事实检查被证明是更便宜的鲁棒性手段。
Another big reason to use combination of frontier models. Chain-of-thought monitoring is treated as...
Another big reason to use combination of frontier models. Chain-of-thought monitoring is treated as a reliable safety layer for agents. This DeepMind-affiliated study shows the layer can be argued out of doing its job. Giving the monitor access to the agent reasoning trace raised approval of harmful actions by 9.5 percent on average, because the scratchpad becomes an extra channel for persuasion. The fix was model diversity. Pairing a Claude 3.7 Sonnet monitor with a GPT-4.1 fact-checker from a different family cut policy-violating approvals by up to 45 percent, versus only 6 percent when one model played both roles. If your oversight rests on one model reading another model reasoning, an adversarial agent can talk its way past it. Cross-family fact-checking is the cheaper robustness lever here. Paper: arxiv.org/abs/2607.08066 Learn to build effective AI agents in our academy: academy.dair.ai 💬 9 🔄 7 ❤️ 38 👀 5872 📊 18 ⚡