想搞懂大模型“思考”和“直接答”到底差在哪？这篇用稀疏自编码器拆解了DeepSeek-R1的推理机制，结论挺反直觉。
该研究用Top-K稀疏自编码器分析DeepSeek-R1-Distill-Qwen-7B在数学题上的中间表征，对比思维链（Thinking）与直接作答（NoThinking）两种模式。结果显示，Thinking模式依赖稀疏高强度特征驱动语言演绎，与题目难度无关；NoThinking模式则呈自适应扩散模式，侧重符号操作。通过抑制激活量最大的三个稀疏特征，发现推理与句法结构紧密耦合，干预会破坏LaTeX和框式答案格式。研究还发现，思维链的连贯性依赖特化特征间的脆弱协调，扰动后出现补偿性过度生成和低信息重复。
Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders
While Large Language Models (LLMs) employing Chain-of-Thought (CoT) exhibit superior reasoning capabilities, the neural mechanisms distinguishing this explicit Thinking mode from direct answer generation (NoThinking mode) remain poorly understood. To deconstruct this cognitive process, we apply Top-K Sparse Autoencoders (SAEs) to the intermediate representations of DeepSeek-R1-Distill-Qwen-7B and examine the model's divergent behaviors across math-solving tasks of three distinct difficulty levels. Observationally, we identify a clear distinction in how the model functions under two reasoning modes: Thinking mode relies on sparse and high-intensity feature activations driving verbal deduction independent of problem complexity, whereas NoThinking mode exhibits an adaptive and diffuse pattern prioritizing symbolic manipulation. Causally, suppressing the three most active sparse features by Total Activation Volume reveals three principles: (i) reasoning and syntactic structure are tightly coupled, as interventions consistently degrade \LaTeX{} and boxed-solution formatting; (ii) Thinking responds to disruption with compensatory over-generation marked by increased metacognitive cues and repetitive, low-information continuations; and (iii) coherent CoT behavior depends on a fragile coordination among specialized features, yielding distinct failure modes under perturbation but a consistently impaired output structure.