PapersSelected

SecPO 提出新防御训练方法,对抗提示注入攻击

SecPO: Principled Adversarial Training for Prompt Injection Security

Why we picked it

提示注入一直很难防,这篇把 Meta-SecAlign 的 ASR 从 78.8% 压到 8.8%,还开源了模型和训练数据,做智能体安全的可以看看。

提示注入被列为 AI 智能体的头号威胁,现有防御最强的 Meta-SecAlign 在自适应攻击下攻击成功率(ASR)仍接近 100%。SecPO 论文指出 DPO 损失以未设防的参考模型为锚点是原因之一,并提出理论上更合理的 SecPO 损失。作者还搭建了高效对抗训练流程,合成 best@240 的强注入输入用于训练。在 AgentDojo 的 AttackerMovesSecond 攻击下,SecPO+AT 的 Qwen3.6-27B 将 ASR@800 从 Meta-SecAlign 的 78.8% 降到 8.8%,同时在覆盖工具调用、指令跟随、科学和数学的九项基准上良性能力没有明显下降。代码、训练数据和模型已在 GitHub 与 Hugging Face 开源。

Original · arXiv cs.AI

SecPO: Principled Adversarial Training for Prompt Injection Security

Recognized as a top threat to AI agents, prompt injection allows instructions from untrusted data (email, documents, websites) to manipulate the agent's operation. The existing best-defended LLM, Meta-SecAlign, is trained with the DPO loss but still suffers from attack success rates (ASRs) approaching 100% under adaptive prompt injections. One reason is that DPO anchors the model around the undefended reference model, which is insecure. We show an idealized reference model---an injection-invariant, utility-preserving oracle---can nevertheless be approximated, as its output distribution in training is interestingly accessible. This new observation motivates a theoretically principled SecPO loss. Another Meta-SecAlign shortcoming is that it trains on weak simulated attacks. We implement an efficient adversarial training (AT) pipeline that synthesizes strong injected inputs (best@240), and train the model on them. Our AT reduces ASRs by 1.5-14.5$\times$ across three SoTA adaptive attacks, compared with Meta-SecAlign; the SecPO loss reduces them by a further 2.4-4.6$\times$ compared with using DPO loss. In agentic workflows, our SecPO + AT Qwen3.6-27B is much more secure than Meta-SecAlign: 8.8% vs. 78.8% AgentDojo ASR@800 under the AttackerMovesSecond attack. Moreover, our defended model shows no noticeable reduction in benign utility in nine benchmarks covering agentic tool calling, instruction following, science, and math. We release our $\href{//github.com/Sizhe-Chen/SecPO}{\text{code}}$, $\href{//huggingface.co/datasets/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO-trainset}{\text{training data}}$ (with optimized injections from AT), and $\href{//huggingface.co/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO}{\text{model}}$.