RCD：用对比解码缓解思维链推理中的捷径依赖
Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding
论文发现模型做 CoT 时会抄近路不看推理步骤，RCD 用对比解码让答案真正依赖推理内容，五个基准准确率都涨，还不用训练。
论文提出 Rationale Contrastive Decoding（RCD），把 context-aware decoding（CAD）应用于 CoT 推理，通过对比有/无 rationale 条件下的下一个 token 分布，放大因 rationale 而提升似然的 token。论文用 KL 正则化的奖励最大化目标给出该解码规则的理论解释。在 5 个推理基准上，RCD 的答案准确率超过标准 CoT。使用 ground-truth 和随机 rationale 的干预分析显示，RCD 提高了最终答案对 rationale 内容的行为依赖。该方法无需训练。
Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding
While Chain-of-Thought (CoT) prompting improves the reasoning performance of large language models (LLMs), recent studies have shown that final answers do not always depend on the generated rationales. In particular, LLMs may rely on query-specific shortcuts rather than the information provided by their reasoning steps. In this paper, we investigate whether context-aware decoding (CAD), an existing contrastive decoding method, can mitigate such shortcut behavior when applied to CoT reasoning. We study this application as Rationale Contrastive Decoding (RCD), which contrasts the next-token distributions conditioned on the query with and without its rationale, amplifying tokens whose likelihood increases due to the rationale. We provide a theoretical interpretation of this decoding rule via a KL-regularized reward-maximization objective. Experiments across five reasoning benchmarks demonstrate that RCD improves answer accuracy over standard CoT. Furthermore, intervention analyses using ground-truth and random rationales reveal that RCD increases the behavioral dependence of final answers on the rationale content. Our findings suggest that contrastive decoding offers a simple, training-free approach to mitigating query shortcuts.
