论文精选

tcGP:面向贝叶斯优化的目标导向高斯过程下尾校准方法

Goal-Oriented Lower-Tail Calibration of Gaussian Processes for Bayesian Optimization

精选理由

做贝叶斯优化的团队终于有了专门解决下尾校准问题的工具——tcGP直接提升低值区域的预测可靠性,让EI采样更精准,做超参数调优或实验设计的建议试试。

AI 摘要

贝叶斯优化依赖高斯过程预测分布来选择评估点,但核函数和超参数选择可能导致预测分布校准不良,影响探索-利用平衡。针对最小化问题,期望改进等采样准则依赖于当前最优值以下的预测分布,下尾校准直接决定采样决策。本文提出tcGP,一种后处理方法,专门校准高斯过程在低阈值以下的预测分布,并证明基于tcGP的EI全局优化算法在设计空间中是稠密的。标准基准实验表明,tcGP相比标准GP和全局校准GP,显著改善了下尾校准和贝叶斯优化性能。

原文 · arXiv cs.LG

Goal-Oriented Lower-Tail Calibration of Gaussian Processes for Bayesian Optimization

Bayesian optimization (BO) selects evaluation points for expensive black-box objectives using Gaussian process (GP) predictive distributions. Kernel choice and hyperparameter selection can lead to miscalibrated predictive distributions and an inappropriate exploration-exploitation trade-off. For minimization, sampling criteria such as expected improvement (EI) depend on the predictive distribution below the current best value, so lower-tail miscalibration directly affects the sampling decision. This article studies goal-oriented calibration of GP predictive distributions below a low threshold $t$ in the noiseless setting, for standard GP models with hyperparameters selected by maximum likelihood. A framework for predictive reliability below $t$ is introduced, based on two notions of spatial calibration: occurrence calibration over the design space and thresholded $μ$-calibration on sublevel sets of the form $\{x\in\mathbb{X}, f(x)\le t\}$. Building on this framework, we propose tcGP, a post-hoc method that calibrates GP predictive distributions below~$t$, and we show that the resulting EI-based global optimization algorithm remains dense in the design space. Experiments on standard benchmarks show improved lower-tail calibration and BO performance relative to standard GP models and globally calibrated GP models.