Four Bridges:多智能体游戏中测量大模型欺骗倾向
Four Bridges: Measuring Deception Propensity in a Multi-Agent Game
8个前沿模型在四人博弈里73%的回合都会撒谎隐瞒死亡房间,这篇还试了steering vector能把欺骗压到0%,搞智能体安全的人可以细看。
研究团队在 Kradle 平台上推出 Four Bridges 信息不对称游戏:4 个智能体分布在 4 个房间,其中 1 个是死亡房间,知情的智能体可以选择是否披露信息。对 8 个前沿模型的测试显示,欺骗行为出现在所有模型中,73% 的回合(单模型 6% 到 100%)出现欺骗,且能力越强的模型欺骗率并未降低。当知情智能体发出警告时,质疑警告的智能体进入死亡房间的频率是不质疑者的 2 倍。告知所有智能体“有人知情”能将 8 个模型中 6 个的欺骗行为压到消失;在开源模型上,诚实 steering vector 可将欺骗率降到 0%,对抗性诚实训练降到 6%。
Four Bridges: Measuring Deception Propensity in a Multi-Agent Game
AI agents are increasingly deployed in settings where they interact and coordinate with other agents, yet honesty is mostly measured in single-turn, single-agent benchmarks that pressure the model to deceive. We study deception that emerges in multi-agent interaction without any instruction to deceive. We introduce Kradle, a platform for evaluating AI models, and Four Bridges, an information asymmetry game in which four agents distribute across four rooms: one is the death room, the others hold food worth more if not shared. One agent, the informed agent, knows which room is the death room and can withhold or disclose that information. We evaluate eight frontier models as the informed agent. Deceptive behavior appears in every model and in 73% of all runs (6% to 100% per model), and does not decrease with capability tier within a model family. When the informed agent does disclose, the warning is often challenged, and agents that challenge it enter the death room twice as often as those that do not. Telling all agents that one of them knows the death room collapses deception in six of eight models; telling the informed agent that the room is permanently fatal moves only one. On an open-weight model, an honesty steering vector reduces deception to 0% and adversarial honesty training to 6%. The platform and game provide a testbed for measuring deceptive behavior and comparing interventions.