做LLM推理优化的团队终于有了一个能同时提升准确率和缩短响应长度的方案——CES框架在数学推理任务上效果显著，建议做模型推理效率的开发者点开看看具体实现。
针对大语言模型推理中响应长度与准确率的权衡问题，研究者提出条件熵塑形（CES）框架。CES基于DAPO，利用token级熵作为不确定性信号，对正确推理路径的高熵“分叉点”进行惩罚以提升简洁性，对错误路径的高熵点给予奖励以鼓励探索。在DeepSeek-R1-Distill-7B上测试12个数学基准，CES在保持或提升准确率的同时显著缩短响应长度，在1.5B小模型和域外基准上也有类似效果。该方法为自适应推理提供了新思路，尤其适合需要高效准确推理的场景。
Taming the Thinker: Conditional Entropy Shaping for Adaptive LLM Reasoning
Entropy-based deep reasoning has emerged as a promising direction for improving the reasoning capabilities of Large Language Models (LLMs), but existing methods often either increase response length indiscriminately or shorten responses at the cost of accuracy. To better balance this trade-off, we introduce Conditional Entropy Shaping (CES), a framework that dynamically controls token-level response entropy, enabling LLMs to produce concise solutions on simple problems while encouraging deeper exploration on hard ones. Built on DAPO, CES uses token-level entropy as an uncertainty signal and applies a conditional bidirectional policy: it penalizes high-entropy "forking point" tokens on correct reasoning paths to improve conciseness, and rewards them on incorrect paths to encourage exploration and error correction. We implement CES on DeepSeek-R1-Distill-7B and evaluate it on 12 mathematical benchmarks. CES consistently improves average accuracy while reducing response length relative to DAPO, and supplementary experiments show similar trends on a smaller 1.5B backbone and on out-of-domain benchmarks.