Ribbon:可扩展的近似与鲁棒不确定性量化

Ribbon: Scalable Approximation and Robust Uncertainty Quantification

精选理由

Ribbon用一种巧妙的线性化技巧,让你不用反复训练就能估算模型的不确定性,而且结果和更复杂的bootstrap差不多。

AI 摘要

Ribbon是一种针对Dirichlet重加权bootstrap的可扩展近似方法,通过单次拟合模型后的影响函数线性化替代重复重拟合。该方法仅需事后线性代数运算即可保持贝叶斯bootstrap的一阶数据重加权结构,并可通过验证数据调整不确定性尺度。在合成回归、MNIST分类和California Housing基准测试中,Ribbon在避免重复模型训练的同时提供了有竞争力的预测性能和改进的校准效果。

原文 · arXiv cs.LG

Ribbon: Scalable Approximation and Robust Uncertainty Quantification

Reliably quantifying predictive uncertainty is difficult for complex, high-dimensional, or misspecified models. Both fully Bayesian and bootstrap resampling methods provide principled uncertainty estimates but are often too expensive for modern machine-learning models because they require posterior sampling or repeated model refitting. We introduce Ribbon, a scalable approximation to Dirichlet-reweighted bootstrap uncertainty. Ribbon replaces repeated refitting with an influence-function linearization around a single fitted model, preserving the first-order data-reweighting structure of the Bayesian bootstrap while requiring only post-hoc linear algebra. Ribbon approximates the Bayesian-bootstrap or weighted-likelihood-bootstrap refitting target. With a general concentration parameter, Ribbon gives a calibrated Dirichlet-reweighting family whose uncertainty scale can be tuned on validation data. We show that Ribbon is asymptotically equivalent to a flat-prior Laplace approximation under correct likelihood specification and recovers the robust sandwich covariance under misspecification. Across synthetic regression, MNIST classification, and California Housing benchmarks, Ribbon provides competitive predictive performance and improved calibration in several settings while avoiding repeated model retraining.