这篇论文提出了AD-CERT，用对抗蒸馏帮模型在保持准确率的同时更容易通过形式化验证，在多个基准上刷新了认证精度记录。
AD-CERT是一种新的认证训练方法，结合了对抗蒸馏与区间边界传播（IBP）上界。该方法在多个鲁棒性基准上实现了最先进的认证性能。在统一设置下，逻辑级对抗蒸馏相比鲁棒特征空间蒸馏，认证准确率提升了高达5.40个百分点。论文通过优化最坏情况损失的上界与下界，改善了标准准确率与认证准确率的权衡。
Improving Certified Robustness via Adversarial Distillation
Certified training aims to produce models whose predictions can be formally verified against adversarial perturbations, typically by optimising upper bounds on the worst-case loss over an allowed perturbation set. For neural networks, certified training methods based purely on tight relaxation bounds produce networks that are amenable to certification, but sacrifice standard accuracy. Conversely, adversarial training often yields stronger empirical robustness and standard accuracy, but the resulting models are generally difficult to certify with neural network verifiers. Recently, the literature has shown that better standard-certified accuracy trade-offs can be achieved by combining adversarial training objectives with loose over-approximations based on Interval Bound Propagation (IBP), effectively interpolating between lower and upper bounds of the worst-case loss. Building on this, we introduce AD-CERT, a certified training objective that combines adversarial distillation with an IBP upper bound. We show that distilling adversarial information over the logit space from an empirically robust teacher provides an effective lower bound surrogate for certified training, with AD-CERT achieving state-of-the-art certified performance on several robustness benchmarks. Furthermore, in a unified setup, distilling adversarial information at the logit-level is shown to improve certified accuracy over a robust feature-space distillation objective by up to 5.40 percentage points.