研究用贝叶斯模型评估 DeepSeek-R1-Distill 系列：规模越大能力越强但效率不变
Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models
这篇用分层贝叶斯方法量化了 DeepSeek-R1-Distill 系列的推理能力与 token 消耗，结论是加大模型只提升能力不提升效率，做选型或研究 scaling 的朋友可以看看。
一项研究用分层贝叶斯模型评估 DeepSeek-R1-Distill 系列在四类算术与算法推理任务上的能力与效率。结果显示，固定模型规模下，答对概率随实例规模近似指数衰减，衰减尺度随模型规模亚线性增长，说明能力提升收益递减。输出长度随实例规模呈幂律增长，但幂律参数不随模型规模系统变化，即更大模型并未变得更省 token。研究据此指出单纯堆规模这一策略的局限。
Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models
Capability and efficiency are two key dimensions of reasoning in large language models (LLMs). Capability refers to the ability to solve a given problem correctly, whereas efficiency refers to the ability to do so with limited resources. When LLMs use Chain-of-Thought (CoT) reasoning to solve problems of controlled hardness, both the number of problems solved correctly and the number of tokens required to reach a correct answer depend on problem hardness and model size. However, how these factors jointly shape capability and efficiency remains poorly understood. Here, we use hierarchical Bayesian models to evaluate the capability and efficiency of LLMs from the DeepSeek-R1-Distill model family across four classes of arithmetic and algorithmic reasoning problems. At a fixed model size, the probability of correctly solving an instance decays approximately exponentially with instance size, our proxy for problem hardness. The decay scale grows sublinearly with model size, indicating that larger models are more capable, but that capability gains diminish with scale. Output length grows as a power law with instance size, which serves as a proxy for difficulty. However, the parameters of this power law do not vary systematically with model size, suggesting that larger models do not become more efficient. Together, these findings reveal potential limitations of naive scaling as a strategy for developing more capable AI systems: capability improves with diminishing returns, while efficiency shows little to no improvement.
