这项研究揭示了CoT推理中大量步骤是“无用功”,做推理模型优化或长链推理应用的开发者可以直接用早退策略节省计算成本,值得关注。
该研究通过早期退出方法估计链式推理中每一步的因果重要性,发现推理过程存在一个“承诺边界”——模型在单个步骤中从临时猜测突然转向稳定、高置信度的最终答案。此边界之后的所有推理步骤(称为“表观CoT”)对最终答案概率无影响。利用注意力探针,研究人员能从中间步骤高精度线性解码答案形成阶段,并泛化到未见任务。基于此信号,模型可在承诺边界提前退出推理块,平均缩短55%的推理链长度,且性能几乎不受影响。
Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
Chain-of-thought (CoT) reasoning is the dominant paradigm for inference-time scaling in language models, yet the causal influence of individual steps on the final answer poorly understood. We estimate each step's causal importance via early exit and use this measure to study how answers form across the reasoning traces of several model families. Across diverse tasks, we find that reasoning typically crosses a \emph{commitment boundary} -- a sharp transition from transient intermediate guesses to a stable, high-confidence answer. This transition often happens in a single step, well before the model's reasoning block ends, and is followed by \emph{epiphenomenal} CoT steps that leave the final answer probability unaltered. Using attention probes, we show that answer-formation stages can be linearly decoded from intermediate reasoning steps with high accuracy and generalize robustly to unseen reasoning tasks. We exploit this signal to early-exit reasoning blocks at the commitment boundary, reducing the length of CoTs up to 55\% on average with negligible impact on model performance.