CLORE 解决了推理模型输出冗长、重复的痛点，做推理优化或部署长链模型的团队可以直接参考其内容级编辑方法，比单纯限制长度更精细。
CLORE 是一种针对大语言模型推理效率的内容级优化框架。它通过外部增强模型对正确的推理轨迹进行编辑，删除重复、不清晰或任务无关的内容，以及答案确定后的多余推理，同时保留最终答案。编辑后的轨迹与原始轨迹构成对比对，通过无参考的 DPO 目标与标准策略梯度训练联合优化。在 DeepSeek-R1-Distill-Qwen-7B 和 Qwen2.5-Math-7B 上的数学推理基准测试中，CLORE 提升了准确率与效率的平衡，并与 GRPO、DAPO 等方法兼容。内容级分析表明，CLORE 能有效减少重复推理、不清晰内容和答案后的探索，为长度级控制提供了互补方向。
CLORE: Content-Level Optimization for Reasoning Efficiency
Reinforcement learning post-training has improved the reasoning ability of large language models, but often produces unnecessarily long, repetitive, or semantically opaque reasoning traces. Existing efficient reasoning methods mainly regulate response length through explicit budgets or length-aware rewards, leaving intermediate reasoning content weakly supervised. We propose CLORE, a content-level optimization framework that improves reasoning efficiency by editing correct on-policy rollouts. CLORE uses an external augmentation model to delete repetitive segments, illegible or task-irrelevant content, and superfluous reasoning after the solution is established, while preserving the final answer. The resulting augmented--original pairs are optimized with an auxiliary reference-free DPO objective alongside standard policy-gradient training. By restricting augmentation to correct trajectories and performing local deletion, CLORE keeps edited rollouts close to the policy distribution and mitigates off-policy mismatch. Experiments on DeepSeek-R1-Distill-Qwen-7B and Qwen2.5-Math-7B across five mathematical reasoning benchmarks show that CLORE improves the accuracy--efficiency trade-off and remains compatible with GRPO, DAPO, Training Efficient, and ThinkPrune. Content-level analyses further show that CLORE reduces repetitive reasoning, illegible content, and post-answer exploration, supporting content-level supervision as a complementary direction to length-level control.