大模型安全对齐的脆弱性一直是部署痛点，AIR用巧妙的锚定策略解决了“一改措辞就破防”的问题，做安全对齐的团队可以直接集成到现有训练流程中。
论文指出当前大模型的安全对齐存在脆弱性：同一恶意意图换种措辞就可能绕过防护。作者提出“上下文不变性对齐”概念，要求模型行为基于底层意图而非表面形式。为此设计了锚定不变性正则化（AIR），将可验证提示作为锚点，仅对开放变体进行正则化，避免降低可靠变体的性能。在安全、道德推理和数学任务上，AIR将分布内准确率提升12.71%，分布外一致性提升33.49%，使安全约束对对抗性措辞更鲁棒。该方法可作为插件与GRPO等偏好优化方法结合使用。
Towards Context-Invariant Safety Alignment for Large Language Models
Preference-based post-training aligns LLMs with human intent, yet safety behavior often remains brittle. A model may refuse a harmful request in a standard prompt but comply when the same intent is wrapped in adversarial wording. We suggest that robust safety requires context-invariant alignment, where behavior depends on the underlying intent rather than surface form. Enforcing invariance is difficult in alignment because not all training signals are equally trustworthy; for some prompt variants we can obtain verifiable feedback (e.g., multiple-choice), while for open-ended variants we typically rely on noisy, gameable reward proxies (e.g., learned judges). As a result, standard symmetric invariance regularizers can reduce cross-context discrepancies by lowering performance on reliable variants instead of improving open-ended robustness. To address this, we introduce Anchor Invariance Regularization (AIR), which treats verifiable prompts as anchors and uses a stop-gradient target to regularize only the open-ended variants toward the anchor performance. AIR is implemented as a plug-in auxiliary loss and combined with group-based preference optimization (e.g., GRPO) via heterogeneous prompt grouping. Across Safety, Moral Reasoning, and Math, AIR improves context invariance, boosting in-distribution group accuracy by 12.71% and out-of-distribution consistency by 33.49%, making safety constraints robust to adversarial framings.