对抗性微调中的预防性转向研究
Active Adaptation, Not Static Defense: Temporal Dynamics of Preventative Steering in Adversarial Fine-Tuning
清华团队发现预防性转向依赖主动适应而非静态防御，提出新方法提升模型安全性。
研究人员分析了预防性转向(Preventative Steering)在大型语言模型中的时间动态机制。该技术通过在微调时注入不良特征向量并在评估时移除它们。研究发现，防御机制源于早期的补偿性适应阶段，随后进入校正信号衰减的稳态阶段。基于此，研究者提出了渐进强度调度(PIS)，在Qwen2.5和Gemma-3模型上提升了安全鲁棒性。
Active Adaptation, Not Static Defense: Temporal Dynamics of Preventative Steering in Adversarial Fine-Tuning
Large language models remain fragile against malicious fine-tuning, motivating training-time defenses against harmful persona drift. Preventative Steering injects undesirable-trait persona vectors during fine-tuning and removes them at evaluation time, yet the mechanism behind its lasting protection remains unclear. Analyzing its temporal optimization dynamics, we find that the defense emerges from an early compensatory adaptation phase followed by a steady-state phase where the corrective signal decays; in parameter space, attention output projections emerge as the dominant residual-write route for defensive updates. Through Intervention Delta Preservation (IDP) and IDP Continuation experiments, we further show that preserving or reinjecting the weight offset fails to maintain protection, indicating that preventative steering relies on active adaptation rather than a static defense. Motivated by this finding, we propose Progressive Intensity Scheduling (PIS), which starts with a moderate injection strength and increases it after static-strength alignment begins to decay. Across the evaluated Qwen2.5 and Gemma-3 models, PIS improves safety robustness over static-strength steering while reducing harmful trait expression.