ReliableNet:用机会约束实现可信分类,控制自信错误率

ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning

精选理由

想减少模型自信犯错?ReliableNet 直接约束这种错误率,比校准和选择性预测更稳,测试里表现最好。

AI 摘要

ReliableNet 提出一种新的训练方法,直接约束模型同时自信且预测错误的联合概率(JCW),使其低于用户设定的风险预算 α。该方法将问题建模为机会约束的经验风险最小化,并用保守光滑内近似保证可行性。在四个表格和两个图像数据集上,ReliableNet 是唯一在所有分布内数据集和随机种子下满足 JCW 预算的方法。在多种分布偏移下,它取得最低的经验 JCW,同时保持较高的准确率和校准性能。

原文 · arXiv cs.LG

ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning

A prediction that is both confident and wrong is a critical reliability failure because it can bypass abstention and human review precisely when the model is mistaken. Empirical risk minimization (ERM) controls average loss but not this failure directly, while calibration, uncertainty estimation, conformal risk control, and selective prediction methods target related reliability properties rather than bounding the joint failure event during training. We propose ReliableNet, which constrains the Joint Confident-Wrong (JCW) probability, the probability that a prediction is simultaneously confident and incorrect, below a user-specified risk budget $α\in(0,1)$. We formulate this as a chance-constrained ERM problem, use a conservative smooth inner approximation whose population feasibility implies the original JCW constraint. Across four tabular and two image datasets, ReliableNet is the only method certified within the JCW budget for every dataset and seed in distribution, when compared against baselines spanning ERM, post-hoc calibration, conformal risk control, and selective prediction. Under demographic, ambiguity, spurious-correlation, novel-class, and covariate shifts, it achieves the lowest empirical JCW among the compared methods while remaining very competitive in accuracy, coverage, calibration, and selective prediction. Risk-coverage results further indicate that ReliableNet achieves better selective ranking than the benchmark methods on most datasets. Overall, ReliableNet provides a principled approach to trustworthy classification.