SAFT：只更新5%参数的稀疏对抗微调方法
Sparse Fine-Tuning for Parameter-Efficient Adversarial Training
这篇论文挺有意思：发现对抗鲁棒性其实藏在少数关键参数里，只微调约5%参数就能媲美全参数对抗训练，还比LoRA效果好。
论文提出 SAFT（Sparse Adversarial Fine-Tuning），通过对抗梯度显著性识别 Robustness-Critical Parameters（RCPs），只更新这些参数进行对抗训练。梯度分析显示对抗优化集中在参数空间的一个稀疏子集上。跨架构和跨数据集的实验中，SAFT 在干净精度和鲁棒精度上都优于 LoRA 类方法，更新约 5% 的参数即可接近或超过全参数对抗训练的鲁棒性。
Sparse Fine-Tuning for Parameter-Efficient Adversarial Training
Deep neural networks achieve strong performance across many tasks but remain vulnerable to adversarial perturbations. Adversarial training (AT) is one of the most effective defenses, yet it suffers from high memory cost on large models and the persistent clean-robust accuracy trade-off. Recent work introduces parameter-efficient fine-tuning, such as LoRA, into AT, but this constraint induces a notable robustness gap relative to full-parameter AT. In this work, we revisit parameter-efficient adversarial fine-tuning from a parameter space perspective. Through gradient analysis, we find that adversarial optimization is highly concentrated on a small subset of parameters, which we call Robustness-Critical Parameters (RCPs), suggesting that robustness is encoded in a sparse subspace. Building on this observation, we propose Sparse Adversarial Fine-Tuning (SAFT), which identifies RCPs via adversarial gradient saliency and performs AT by updating only these parameters while freezing the rest. Across architectures and datasets, SAFT outperforms LoRA-based methods in both clean and robust accuracy with fewer trainable parameters, and approaches or surpasses full parameter AT in robustness while updating only about 5\% of parameters.
