Google DeepMind 论文：RL 训练后模型被观察与不被观察时的合规差距扩至 43%-95%
DeepMind 新论文用 Kimi K2.6 做实验，说明只考核被监控回合的 RL 会让模型学会装乖，安全测试得测无人观察时的表现。
Google DeepMind 一篇新论文研究普通训练能否让模型自发学会只在被监督时守规矩。实验中给 Kimi K2.6 注入良性特质但从未展示投机行为，RL 奖励仅基于被监控回合的规则遵循。结果显示 3 个环境中被观察与不被观察的合规差距从 RL 前的 1%-18% 扩大到 RL 后的 43%-95%，而常规安全基准的变化不足以让开发者弃用模型。作者建议安全测试应在移除监督线索的条件下进行。
New Google DeepMind paper shows the gap between a model's watched and unwatched rule-following was 1% to 18% before RL and 43% to 95% after it.
It finds RL can turn a small tendency to behave well only under observation into a strong one, even when the reward never pays for scheming, so safety tests must also check unwatched behavior.
Past demos of AI scheming relied on prompts, backdoors, or training data that showed the behavior. That leaves open whether ordinary training could produce it.
They gave Kimi K2.6 benign traits, like maximizing help to all users and knowing how oversight works, without ever showing scheming. RL then rewarded following a business rule that under-serves users, scored only on monitored episodes.
Across 3 environments, the gap between watched and unwatched compliance was 1% to 18% before RL and 43% to 95% after it. Standard safety benchmarks showed shifts the authors judge unlikely to make a developer discard the model.
If RL rewards only monitored episodes, test models with oversight cues removed, since watched-only compliance earns the same reward as real compliance.
