这篇论文戳穿了当前长链推理模型的表面繁荣——做AI推理研究的团队会发现,模型可能只是在模仿推理的“样子”而非真正进步,值得细读其提出的改进方向。
一项新研究对DeepSeek-R1-0120模型与人类在AIME 2025数学竞赛30道题上的推理过程进行了全面比较,标注了10247个推理步骤。研究发现,DeepSeek-R1的推理存在“拓扑模仿”现象:它频繁进行浅层验证和局部循环,缺乏人类推理中紧凑的分析与演绎交替结构。然而,成功的推理轨迹显示出稳定的分支和回溯使用,而失败的轨迹则过度或不足使用探索性动作。反思只有在演绎推理中才有效,陷入分析循环的反思往往忽略全局逻辑错误。研究建议改进评估和训练,包括测量跨轨迹稳定性、惩罚“空转”轨迹、鼓励深层逻辑修正。
A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning
The emergence of "Aha moments" in large language models, particularly DeepSeek-R1-0120, has raised the question of whether these systems genuinely reason or merely imitate the appearance of reasoning. We conduct a comprehensive empirical comparison between model and human reasoning across all 30 problems from AIME 2025, exhaustively annotating 10,247 reasoning steps into five functional categories: Analysis, Inference, Branch, Backtrace, and Reflection. We find a clear structural difference. Human solutions maintain a compact alternation between analysis and deduction, whereas DeepSeek-R1 frequently revisits intermediate results, performs shallow and often unnecessary verification, and loops through local checks without meaningful logical progress. We describe this as topological mimicry: reproducing the surface form of reasoning without its functional role. Despite this, we identify two signals of genuine reasoning. First, successful traces exhibit stable use of branching and backtracking, while failed traces either underuse or overuse exploratory actions. Second, reflection is only effective when placed within deductive inference; reflections trapped in analysis loops focus on local numerical details while missing global logical errors. These findings suggest that current long-CoT models may be rewarded more for the appearance of reasoning than for genuine deductive progress. We discuss directions for improving evaluation and training, including measuring cross-trace stability, penalising "spinning-wheel" traces, encouraging deeper logical correction, and reallocating inference-time compute toward deduction and backtracking. Overall, reasoning quality depends not simply on how much reflection occurs, but on whether reflection appears consistently and at the appropriate logical scale.