AI模型精选

Adaptive Bayesian Online Learning via Expert Aggregation

精选理由

这篇论文用专家聚合让贝叶斯在线学习自适应,无需事先设定学习率或先验,实验还证明能自动跟踪最强专家。

AI 摘要

论文提出一种通过专家聚合的自适应贝叶斯在线学习方法,将贝叶斯更新规则视为专家,根据序贯预测损失进行聚合。理论证明该聚合与事后最优专家竞争,聚合成本取决于每个专家每轮性能评估方式。在在线共形推断中,该方法得到长期随机覆盖率的平滑贝叶斯版本;在高斯过程回归中,累积预测KL风险达到最优,且自适应于未知Hölder光滑度(对数因子)。实验表明该聚合无需先知专家选择即可跟踪强专家。

原文 · arXiv cs.LG

Bayesian online learning promises uncertainty-aware prediction on data streams, but its performance hinges on inferential choices, including learning rates, prior distributions and variational families, which are usually fixed before seeing the stream. We address this by treating Bayesian update rules as experts and aggregating the Bayesian experts according to sequential predictive losses. We prove that the resulting aggregate competes with the best expert in hindsight at an aggregation cost determined by how each expert's per-round performance is evaluated. We instantiate the framework in online conformal inference and Gaussian process regression. The conformal inference application yields a smoothed Bayesian counterpart of adaptive conformal inference with long-run randomized coverage, while the Gaussian process application gives an oracle inequality in cumulative predictive Kullback-Leibler risk and adaptation to unknown Hölder smoothness up to logarithmic factors. Experiments show that the aggregate tracks strong experts without oracle expert selection.