Papers

Evolution Under Fire:用自适应攻击压力测试自动进化智能体安全防护

Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

Why we picked it

做智能体安全的人该看看这篇:它不用手写防御规则,而是让安全 harness 自己在攻击中进化。GPT-4o-mini 上把攻击成功率从 50.4% 打到 3.0%,还能跨模型迁移到 Qwen3.5-35B-A3B。比静态防御方案实际可用性强。

论文提出 Evolution Under Fire,一个针对工具调用型智能体的安全防护自动进化框架。核心问题在于恶意指令和正常任务指令会通过同一个交互通道(邮件、网页、文档)进入智能体,而现有防御多为手工设计、静态固定,改进依赖反复人工重写。框架分两个阶段交替进行:一边用效用约束的 Pareto 调度树搜索,从执行证据中自动演化出完整可执行的安全 harness,期间保持任务模型、基准、工具和评估器不变;另一边用自适应攻击压力进化,自动发现能保留正常功能的攻击样本来指导下一轮 harness 进化。

Body

实验结果给出具体数字:在 AgentDyn 上配合 GPT-4o-mini,进化后的 harness 将攻击成功率从 50.4% 降到 3.0%,同时干净任务和受攻击任务的效用都超过初始 harness,安全-效用权衡优于复现的基线防御。在 Agent-SafetyBench 上配合 Qwen2.5-7B-Instruct,攻击成功率从 82.6% 降到 62.0%,且保留了任务完成度和运行有效性。进化出的 harness 还能免再训练直接迁移到 Qwen3.5-35B-A3B,攻击成功率从 32.1% 降到 0.9%,受攻击任务效用反而提升。

Original · arXiv cs.AI

Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

Tool-using language-model agents must act on information from untrusted emails, webpages, and documents. This creates a central security challenge: malicious instructions and legitimate task directives can arrive through the same interaction channel. Existing defenses are largely hand-designed and static, making further improvements dependent on repeated manual redesign. We introduce Evolution Under Fire, a staged framework that couples safety-harness evolution with adaptive stressor evolution. Utility-constrained, Pareto-scheduled tree search evolves complete executable safety harnesses from execution evidence while keeping the task model, benchmark, tools, and evaluator fixed. Adaptive stressor evolution then discovers utility-preserving attacks that guide a subsequent stage of harness evolution. On AgentDyn with GPT-4o-mini, an evolved harness reduces attack success from 50.4% to 3.0%, improves clean and attacked utility over the initial harness, and offers a stronger security-utility trade-off than reproduced defenses. On Agent-SafetyBench with Qwen2.5-7B-Instruct, harness evolution reduces attack success from 82.6% to 62.0% while retaining helpfulness and runtime validity. Evolved harnesses also transfer without further evolution to Qwen3.5-35B-A3B, reducing attack success from 32.1% to 0.9% while improving attacked-task utility.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include agents, AI safety, GPT-4o-mini, Qwen2.5-7B-Instruct.

Source attribution: curated from arXiv cs.AI (@Yujun Wang, Yilun Liu, Jinhe Bi, Krishna Kanth Nakka, Xuebing Zhou, Hinrich Sch{ü}tze, Yunpu Ma).