论文：自监督置信度微调让推理模型生成 token 减少 25%
Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
只用600道题微调，不训练停止机制，就让Qwen、Gemma这些模型推理时少生成25%的token，思路挺有意思。
这篇 arXiv 论文提出用置信度作为训练目标来提升推理效率。作者仅用 600 道训练题对推理模型做自监督微调，让模型在推理轨迹的中间点预测自己对答案的置信度，训练损失中不包含任何关于推理长度或停止的目标。在 Gemma、Qwen、Nemotron、GPT-OSS 四个模型上，数学、科学、编程推理基准测试中，准确率持平的情况下生成 token 最多减少 25%。这一效率提升与显式优化更短推理的方法相当，且分析显示置信度监督基本保留了基座模型的推理结构。
Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency either through inference-time early-stopping mechanisms or by explicitly encouraging shorter reasoning during training, for example through reinforcement learning with length penalties. We show that substantial efficiency gains can instead emerge from a different kind of supervision: \textit{confidence}. Using a self-supervised procedure, we fine-tune reasoning models to predict their confidence in the answer at intermediate points along their own reasoning trajectories using only 600 training problems. Confidence is used only as a training target: the loss contains no objective for reasoning length, efficiency, or stopping. At inference, the fine-tuned models use the standard generation procedure, with no confidence elicitation or early-stopping mechanism. Despite this, self-supervised confidence fine-tuning makes reasoning more efficient, reducing generated tokens by up to 25\% at matched accuracy across Gemma, Qwen, Nemotron, and GPT-OSS models on mathematical, scientific, and coding reasoning benchmarks, with efficiency gains comparable to methods that explicitly optimize for shorter reasoning. Analysis of reasoning episodes further shows that confidence supervision largely preserves the base models' high-level reasoning composition rather than selectively suppressing particular behaviors. Our results suggest that efficient reasoning may emerge as a downstream consequence of learning metacognitive signals, without being directly optimized.
