研究提出 Self-Instructed CoT 评估:推理模型自我提示可显著提升思维链可控性
Reasoning Models Can Control Their Chains of Thought When They Tell Themselves To
这篇论文发现把约束写进模型自己的思维链里比写在提示里管用得多,做提示词工程或关注 CoT 监控安全的都该看看,尤其是最后那个规避监控的实验。
arXiv 论文提出 Self-Instructed CoT Controllability 评估方法,把约束作为自我指令放进思维链内部而非用户提示。实验显示自我指令下模型对 CoT 的可控性显著高于用户提示方式,即使在提示中加一句提醒遵守约束也能明显改善。论文还演示在监控场景中,模型可利用这种自我指令能力实现 CoT 监控规避,对 CoT monitorability 构成风险。
Reasoning Models Can Control Their Chains of Thought When They Tell Themselves To
Chain-of-thought (CoT) controllability, the ability of reasoning models to control properties of their CoT, is consequential for both their usefulness and reliable oversight. Recent work evaluates this capability by measuring whether models follow CoT constraints given in the user prompt, and reports very low controllability across models. We argue that this low controllability may not reflect an inherent inability of reasoning models to control their CoT, but instead result from naively specifying constraints in the user prompt. To test this, we introduce Self-Instructed CoT Controllability: a complementary evaluation that places the constraint as a self-instruction inside the model's CoT and measures its ability to follow it. We show that, under self-instruction, models achieve significantly higher controllability than with standard user instructions. Even a generic reminder to follow the user's constraint substantially improves controllability. Moreover, effective self-instructions can be bootstrapped into the reasoning through a simple instruction in the user prompt, enabling improved control even without access to the reasoning channel. Finally, using self-instruction, we demonstrate in a realistic monitoring setup that CoT controllability can be leveraged for monitor evasion, posing a risk to CoT monitorability.