这篇论文把校准里的一个坑讲透了:温度缩放能让同一个模型输出任意低的代理误差,但实际错误率根本没变,数值能差几百倍。做不确定性估计的一定要看。
Ishida等人提出的软标签贝叶斯误差估计器β(z)=E[min(z,1-z)]在概率并非真实后验时会产生严重偏差。研究证明温度缩放会导致代理值与真实误差完全脱钩:固定分类器在CIFAR-10、Fashion-MNIST和SVHN的8个二分类任务上,0-1误差不变,代理值可变化56倍至980倍。理论推导表明温度与代理值之间存在连续双射,使同一分类器能报告(0,1/2)区间内的任意代理值。在Logits服从高斯分布时,作者给出了代理-温度曲线的闭式参数解,经验拟合误差小于0.018。校准温度(最小化ECE)与稳定的代理值没有对应关系。
The Calibration Channel Determines the Bayes-Error Proxy: An Exact Law for Temperature-Induced Distortion
The soft-label Bayes-error estimator beta(z) = E[min(z, 1-z)] of Ishida et al. estimates the irreducible error of a binary task directly from probability-valued labels. Recent work by Ushio et al. showed that this estimator is fragile when the probabilities are not the true posterior: even perfectly calibrated soft labels can yield a substantially inaccurate estimate, and they propose isotonic calibration as a consistent remedy. We complement that line of work by characterizing exactly how the most widely used post-hoc calibration map -- temperature scaling -- distorts the proxy. We prove an exact, model-free identity reducing the temperature-scaled proxy to the classifier's margin distribution, from which we obtain (i) strict monotonicity in the temperature and (ii) a continuous bijection from the temperature axis onto the open interval (0, 1/2), so that a fixed classifier -- with fixed decisions and fixed 0-1 error -- can be made to report any proxy value whatsoever. Under a Gaussian model of the logits we further derive a two-parameter closed form for the entire proxy-versus-temperature curve. Across CIFAR-10, Fashion-MNIST, and SVHN (eight binary tasks), the proxy varies by 56x to 980x at constant test error, the closed form reproduces the empirical curve to within 0.018, and the calibration temperature that minimizes the expected calibration error does not coincide with any stable proxy value. Our results give a precise, predictive account of the distortion whose existence motivates calibration-based remedies, and they reinforce the practical recommendation that a proxy value is meaningful only together with the mechanism that produced its probabilities.