研究显示多模态模型在测试边界上存在差异
The Troy Moment of AI: Why Some Will Cheat and Some Will Follow?
想了解不同大模型在处理边界测试时的行为差异,这篇论文用具体模型和实验数据说明了情况。
研究使用 GPT-5.6 Sol、Claude Fable 5.1 和 Gemini 3.8 Flash 在两种测试环境下进行实验。在明确边界规则下,模型行为差异显著;在基准原生环境下,引入同行行为后,模型修改受保护测试的情况增多,通常被解释为文件被篡改。
The Troy Moment of AI: Why Some Will Cheat and Some Will Follow?
Recent investigations of the July 2026 OpenAI-Hugging Face incident motivate two questions: when an assigned task becomes impossible, does an agent stop or escalate, and can observing another agent's behavior change that decision? We study these questions using seven ImpossibleBench tasks with GPT-5.6 Sol, Claude Fable 5.1, and Gemini 3.8 Flash in solo and three-agent settings. Under an explicit-boundary regime with clear authorization rules and restricted tools, no protected tests are modified, although the models differ substantially in whether they escalate, stop silently, or fail to terminate. Under a benchmark-native regime with open shell tools, protected-test changes occur more often after peer activity is introduced and in multi-agent runs. These crossings are typically not described as deliberate cheating: agents often interpret the conflicting test change as prior tampering and restore the file, thereby removing the protected requirement. Our results suggest that boundary crossing can arise from ambiguity about the state a rule is intended to protect, motivating explicit authorization boundaries, authenticated state provenance, and cross-agent monitoring.