这篇论文提出了一种新方法，可以在模型回答之前就预测它的置信度，比传统方法更准更省计算成本。
该论文从时间维度研究大语言模型（LLM）的置信度估计，比较了预解答的Feeling-of-Knowing（FOK）和后解答的Judgement-of-Learning（JOL）在不同的前沿和开源LLM上的表现。实验表明，后解答置信度校准更好、判别力更强，而基于隐藏表示的线性探针能提取比模型显式表达更丰富的置信度信息。作者提出未来置信度蒸馏方法，使用后解答正确性探针作为教师，训练仅基于预解答隐藏表示的预测器。该预测器在仅用预解答信息的情况下恢复了后解答置信度的大部分校准改进，且样本效率高、在同类数据集间可迁移。
Future Confidence Distillation in Large Language Models
Reliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability. Existing approaches, however, largely treat confidence as a property of completed responses, overlooking how confidence-related information evolves throughout the answering process. In this work, we investigate confidence from a temporal perspective by comparing pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) confidence estimates across frontier and open-source LLMs. We show that post-solution confidence is consistently better calibrated and more discriminative than pre-solution confidence, while linear probes trained on hidden representations recover substantially richer confidence-related information than models explicitly verbalise. Building on this observation, we introduce future confidence distillation, which trains predictors operating on pre-solution hidden representations using teacher confidence estimates produced by post-solution correctness probes. Despite requiring only pre-solution representations for inference, distilled predictors recover much of the calibration improvement achieved by post-solution confidence, remain highly sample efficient, and transfer across datasets within the same domain. Together, our findings demonstrate that confidence-related information evolves throughout the answering process and can be anticipated before answer generation is complete, enabling significantly more reliable yet low-cost confidence estimation.