论文精选

Plug-in Losses 简化证据深度学习不确定性估计,含 Softmax 分类器

Plug-in Losses for Evidential Deep Learning: A Simplified Framework for Uncertainty Estimation that Includes the Softmax Classifier

精选理由

做不确定性估计的团队终于有了更简单的实现路径——用标准 Softmax 损失就能获得 EDL 效果,语音识别研究者可以直接替换现有训练流程试试。

AI 摘要

证据深度学习(EDL)通过狄利克雷分布实现单次前向传播的不确定性估计,但其损失函数复杂,难以分析和实现。本文提出一种插件损失(plug-in loss),在狄利克雷均值处评估标准损失(如交叉熵),并证明在温和假设下近似误差随证据增长而衰减。该框架将标准 Softmax 分类器纳入不确定性估计,简化了实现。在 Google Speech Commands 数据集上,该方法在预测准确率和选择性预测性能上与经典 EDL 相当,且更易集成到现有训练流程。这是首次在语音识别任务中通过 EDL 获得覆盖-准确率权衡的实证分析。

原文 · arXiv cs.LG

Plug-in Losses for Evidential Deep Learning: A Simplified Framework for Uncertainty Estimation that Includes the Softmax Classifier

Real-world sensor-based learning systems require uncertainty estimation that is both reliable and computationally efficient. Evidential Deep Learning (EDL) provides single-pass uncertainty estimation by modeling the class probabilities via Dirichlet distributions, where the Dirichlet parameters are predicted by a learned neural network mapping. However, this approach can lead to computational challenges, as Dirichlet expected objectives are more complex than standard supervised learning losses, complicating their analysis and implementation. We address this issue by approximating the objective of the first-order empirical risk minimization problem induced by EDL with a plug-in loss evaluated at the Dirichlet mean and show that, under mild assumptions, the approximation error decays with growing evidence for a broad class of loss functions, including mean-squared error and cross-entropy loss. As a special case, our analysis provides justification for the use of softmax in the context of uncertainty estimation, since under a particular evidence-to-Dirichlet mapping, our framework includes the standard softmax classifier. We validate the proposed simplified objectives on the Google Speech Commands dataset and show that they achieve predictive accuracy and selective prediction performance comparable to classical EDL, while being simpler to implement using standard deep learning losses and training pipelines. To the best of our knowledge, this empirical analysis is the first to obtain coverage-accuracy trade-offs for speech recognition tasks through EDL.