论文揭示 LLM 推理的内部结构：8 种推理操作各有独特模式
这篇论文把推理拆成 8 种操作看模型内部活动，算错了签名还是对的，做对齐和可解释性的人会很有兴趣。
一篇 arXiv 论文发现，LLM 推理的内部状态在生成文本之外存在可识别的结构。研究者追踪了提取事实、分解、回忆、演绎、代数、计算等 8 种常见推理操作，每种操作产生独特的内部模式，且模式在模型中间层最清晰。同一个 token 因承担的推理任务不同，内部表现也不同；屏蔽前 30 个 token 后，下一操作的信号会减弱。关键发现是模型即使算错或推错，内部仍带有正确的操作签名，说明'想做什么'和'做对与否'在内部是可分离的。
The paper finds that LLM reasoning has an internal structure beyond the words being generated, opening a possible route to monitoring reasoning from inside the model.
What an LLM is trying to do and whether it is doing it correctly appear to be separable internally.
The researchers tracked 8 common moves, including extracting facts, decomposition, recall, deduction, algebra, and calculation.
Each move produced a distinct internal pattern, and those patterns were clearest around the middle layers.
Even the exact same token looked different inside the model depending on the reasoning job it was doing.
Context also shaped these states.
When access to the previous 30 tokens was blocked, the signal for the next reasoning operation weakened.
Most importantly, a wrong calculation or deduction could still carry the correct operation signature.
The model can represent what kind of reasoning it is attempting without necessarily getting that reasoning right.
– arxiv. org/abs/2609.04753
Title: "Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs"
