不想重训练模型?PeTeR这个后训练方法不用数据就能给概率电路加固,效果和从头做鲁棒训练差不多。
概率电路(PC)能高效计算复杂联合分布的推理查询,但基于似然的标准训练易对噪声、小样本和分布偏移过拟合。现有分布鲁棒优化需从头训练,而PeTeR提出一种数据无关的后训练框架,无需重新训练即可增强预训练PC的鲁棒性。在多个密度估计基准上,PeTeR有效提升基线模型对随机和对抗扰动的性能,达到或优于依赖数据的鲁棒学习基线。
PeTeR: Post-Training Robustification of Probabilistic Circuits
Probabilistic circuits (PCs) can model complex joint distributions while supporting exact and efficient computation of many inference queries. However, standard likelihood-based PC learning is vulnerable to overfitting and fragile generalization when confronted with data noise, small sample sizes, or distribution shifts. This can be mitigated using distributionally-robust optimization which consider worst-case distributions within a Wasserstein ball of the empirical distribution, but current methods are limited to training a model from scratch in this framework. Instead, we propose PeTeR: a novel, data-free post-training framework designed to robustify pre-trained PCs against distribution shifts without retraining from scratch. Empirical evaluations across multiple density estimation benchmarks demonstrate that PeTeR effectively robustifies baseline models against both random and adversarial perturbations, achieving competitive or superior performance to data-dependent robust learning baselines.