这篇论文揭穿了一个直觉错误：你以为给小白模型喂“学霸笔记”能变强，结果效果还不如它自己瞎写的解题草稿。原因是学霸的思路和它不匹配，硬学反而费劲。
论文在Qwen2.5、LLaMA-3和DeepSeek三个系列上发现：用小模型自身生成并通过拒绝采样选取的轨迹，比用更强Oracle模型精炼的高奖励数据，能更有效提升数学推理。Oracle精炼虽修复逻辑，但引入分布偏移，增加小模型适应成本，抵消了逻辑改进的收益。作者提出风格对齐精炼（Style-Aligned Refinement），保留小模型原生轨迹风格同时融入Oracle逻辑修复，降低适应成本并恢复下游效用。该发现挑战了数学推理蒸馏中依赖奖励模型分数选择数据的常规做法。
The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning
Knowledge distillation from powerful reasoning models is widely used to improve Small Language Models (SLMs) on mathematical reasoning, often assuming that traces with higher reward model scores provide more useful supervision. We identify a counterintuitive \textbf{Quality-Utility Paradox} in mathematical reasoning distillation. Data refined or synthesized by a stronger Oracle obtains higher perceived quality according to reward models, yet consistently underperforms traces generated by the SLM itself and selected through rejection sampling across Qwen2.5, LLaMA-3, and DeepSeek families. Our analysis shows that Oracle refinement couples logical repair with distributional drift away from the SLM's native reasoning distribution. This drift increases the learner's adaptation cost and can outweigh the benefit of improved reasoning logic. To test this mechanism, we introduce \textbf{Style-Aligned Refinement}, which preserves the native trajectory of the SLM while retaining logical repair from the Oracle. This intervention lowers adaptation cost and restores downstream utility. These findings suggest that effective mathematical reasoning distillation should jointly optimize perceived solution quality and learner-data compatibility, rather than relying solely on reward-model scores. The datasets and code are available at https://github.com/Dracoqhl/Quality-Utility-Paradox.