CoT 可控性研究：开头几个 token 决定推理可控程度
论文挺有意思：思维链不听话，问题出在 thinking 标签后的头几个 token，预填一个 token 就能大幅提升可控性。
一项新论文研究思维链（CoT）的可控性问题，发现推理监控之所以困难，部分原因是模型难以控制自己在推理中写什么。研究者将这种限制追溯到 thinking 标签后的最初几个 token：此处熵坍缩，模型会落入少数常见开头。仅预填充一个 token 就能显著提升可控性；禁止高频开头词并加入上下文示例，使多个模型的平均合规度提升 34 到 41 个百分点。补充信息和改写信息的控制能力恢复明显，而遗漏信息的控制响应不一致，两者表现为相互独立的能力。
Nice paper on CoT controllability. They find that chain-of-thought monitoring depends partly on models being unable to control what they write in their reasoning. They traced much of that limit to the first few tokens after the opening thinking tag. Entropy collapses there, and the model falls into a few common openings. Small changes at that point restore a lot of control. Prefilling a single token raised controllability substantially. Banning frequent opening tokens and adding in-context examples raised average compliance by 34 to 41 points across models. These interventions mostly restore control over adding or restyling information. Control over leaving information out responded less consistently. The two kinds behave like separate capabilities. Across models, scores correlate within each group and mostly negatively between groups. arxiv.org/abs/2610.40939 Chat with Paper: academy.dair.ai/papers/the-fir… 💬 35 🔄 14 ❤️ 96 👀 8039 📊 51 ⚡