这篇论文测了11个小模型，发现温度缩放有校准不了的边界，用200道题就能算风险上限，20%风险预算下只有3个敢放手用。
该研究评估了11个指令微调模型在ARC-Challenge和TruthfulQA上的25,168次预测，参数规模从0.5B到14B。理论证明严格单调校准可保持风险覆盖前沿和错误检测AUROC，而温度缩放无法校准置信度恒高于准确率的模型。Clopper-Pearson方法可将200题校准集转化为有限样本风险证书。实证显示8/22个模型-任务对逼近温度缩放不可行下限，Platt缩放将ECE降至0.02。20%风险预算下仅3个模型-任务对获得认证自主权，10%预算下为零，并修复了TruthfulQA多项选择的答案顺序伪影。
Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models
Small open-weight language models increasingly run in private, offline, and cost-sensitive settings, where the key deployment question is not only what a model answers but when it should defer to a human. We study whether verbalized confidence can support risk-controlled deferral, evaluating eleven instruction-tuned models from three families, 0.5B to 14B parameters, on ARC-Challenge and TruthfulQA with 25,168 local predictions. Three theoretical results delimit what calibration can provide: strictly monotone calibration preserves the risk-coverage frontier and error-detection AUROC; temperature scaling cannot calibrate models whose confidence stays above one half while accuracy falls below it; and a Clopper-Pearson procedure converts a 200-question calibration set into a finite-sample risk certificate under an i.i.d. deployment assumption. Empirically, eight of 22 model-task pairs hit the temperature-scaling infeasibility floor within one percentage point of the predicted bound. Platt scaling reduces ECE to as low as 0.02, yet certified autonomy at a 20% risk budget is granted to only three model-task pairs and to none at 10%. We also identify and repair an answer-ordering artifact in the multiple-choice form of TruthfulQA. Calibration gives confidence semantics; certified deferral determines when small models are safe to use.