这篇论文分析了推理模型什么时候该提前停止计算。LearnStop在数学题上比简单规则好,但难题还是用置信度更靠谱。
论文提出LearnStop方法,在固定预算检查点从推理前缀预测正确性,使用答案置信度、熵、前缀投票份额等特征。在GSM8K、MATH-500、MMLU-Pro、AIME-90、GPQA等18个任务-模型设置上评估,涉及Qwen3和DeepSeek-R1蒸馏模型。在GSM8K上使用Qwen3-32B时,后验峰值自适应增益达+0.157,验证选择操作点保持正增益。在多项选择和极难设置上,标量置信度、熵或稳定性规则更具竞争力。结论:学习停止的价值取决于轨迹结构,当许多问题在预算结束前变正确但缺乏可靠标量停止信号时有用。
When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models
Reasoning models spend different amounts of useful computation across instances, but it remains unclear when a learned stopping rule improves over simple confidence or convergence thresholds. We study this question with LearnStop, a hidden-state-free checkpoint stopper for reasoning language models. At fixed budget checkpoints, LearnStop probes a short answer from the current reasoning prefix and predicts prefix correctness from online features such as answer confidence, entropy, prefix vote share, answer stability, and backtracking-marker density. Across 18 task-model settings spanning GSM8K, MATH-500, MMLU-Pro, AIME-90, GPQA, Qwen3, and DeepSeek-R1 distillations, the answer is task-dependent. On free-form math, learned multi-feature stopping improves the fixed-budget frontier and often beats scalar exits: on GSM8K with Qwen3-32B, the empirical frontier reaches a post-hoc peak adapt gain of +0.157, validation-selected operating points preserve positive gains, and the paired gain over the strongest scalar baseline is +0.028. On multiple-choice and very hard settings, scalar confidence, entropy, or stability rules are competitive or stronger. We therefore frame learned stopping not as a universal replacement for scalar exits, but as a tool whose value depends on trajectory structure. We further provide validation-selected operating points, paired bootstrap tests, finite-grid lost-correct risk calibration, cost accounting under KV-fork, prefix-cache, and black-box regimes, H100 serving profiles, checkpoint-schedule sweeps, transfer analyses, and robustness checks. The main practical finding is that learned stopping is useful when many questions become correct before full budget but do not exhibit a single reliable scalar stopping signal; its benefits largely disappear when confidence or answer convergence already solves the stopping problem.