StepGuard:学习级联防护与安全-效用平衡

StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing

精选理由

StepGuard 通过自动数据生成和动态平衡学习,显著提高了安全防护的准确率,同时保持了效用。与 GPT-5.4 相比,防护效果更佳,值得一看。

AI 摘要

StepGuard 是一种级联防护模型,可审计代理轨迹并检查执行前的工具操作。使用 StepGen 生成安全和不安全的轨迹进行训练,Balance-GRPO 动态平衡安全与不安全操作的学习。实验表明,StepGuard 在开放权重防护模型中实现最高平均准确率,与 GPT-5.4 性能相当。在 AgentDojo 和 AgentDyn 上使用 StepGuard,相对于无防护设置,平均攻击成功率降低 77.3%,而平均效用仅下降 2.8 个百分点。

原文 · arXiv cs.AI

StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing

LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage, and unauthorized actions. Existing guardrails often evaluate completed trajectories, leaving pre-execution monitoring of step-level actions underexplored. We propose StepGuard, a step-level guard model that can audit completed agent trajectories and check tool actions before they are executed. To train StepGuard, we introduce StepGen, an automatic data engine that generates safe and unsafe trajectories with the same context but different actions at the risky step. To further reduce over-defense and under-defense, we propose Balance-GRPO, which dynamically balances learning between safe and unsafe actions based on their observed accuracy. Experiments show that StepGuard achieves the highest average accuracy among open-weight guard models, with performance comparable to GPT-5.4. When used to guard agents on AgentDojo and AgentDyn, StepGuard reduces mean attack success rate by 77.3% relative to the no-guard setting, while mean utility drops by only 2.8 percentage points.