Robust CurveMoE Defense Framework
Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity
The method improves adversarial defense performance for computer vision models while reducing training costs, making it valuable for researchers working with limited computational resources.
Robust CurveMoE is a mixture-of-experts framework that connects models specialized for different perturbation norms through a low-loss path. It improves clean, norm-specific, and Union accuracy over MSD and ERMC baselines on CIFAR-100 and ImageNet-100 datasets. The framework achieves 2.37 and 2.13 percentage points higher Union accuracy than the strongest baseline on CIFAR-100 and ImageNet-100 respectively. It uses contribution-guided partial updating to reduce curve-construction cost and selectively expertizes only influential layers.
Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity
Multi-norm adversarial defense aims to protect neural networks against perturbations defined by different norm constraints, but existing methods typically optimize competing robustness objectives within a single parameter configuration, leading to substantial training cost and unfavorable robustness trade-offs. We propose Robust CurveMoE, an efficient mixture-of-experts framework that connects models specialized for different perturbation norms through a low-loss path and exploits the complementary robustness profiles of models along this path. Robust CurveMoE derives clean and norm-specialized experts from robustness-constrained curve locations and selectively expertizes only influential layers, while sharing the remaining parameters across routing paths. To further reduce curve-construction cost, we introduce contribution-guided partial updating, which selects influential curve parameters using initialization-based gradient scores. We also theoretically bound the objective gap between partial and full curve optimization. Experiments on CIFAR-100 and ImageNet-100 with WideResNet and Vision Transformer architectures show that Robust CurveMoE consistently improves clean, norm-specific, and Union accuracy over MSD and ERMC. In particular, it improves Union accuracy by 2.37 and 2.13 percentage points over the strongest baseline on CIFAR-100 and ImageNet-100, respectively. Extensive ablations further validate the effectiveness of partial updating, selective expertization, and robustness-constrained expert selection.
