论文精选73°

工具输出变命令:LLM智能体中的行动诱导与授权分离

When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents

精选理由

研究人员提出SARA框架,解决了LLM智能体中工具输出被滥用的安全问题,在四个主要评估设置中表现优异。

AI 摘要

SARA系统解决了工具增强LLM智能体中工具输出从数据变为命令的安全风险。该系统将行动诱导与执行授权分离,在AgentDojo和AgentDyn测试中,将ASR控制在0.63%以下。SARA通过上下文隔离的行动探测和No-History-Promotion技术,保持任务效用同时提升安全性。

原文 · arXiv cs.AI

When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents

Tool-augmented LLM agents must rely on untrusted runtime Observations to complete open-ended tasks; however, when tool outputs no longer merely provide data but begin to specify concrete actions, they effectively become ``commands'' that can drive real-world side effects beyond user intent. We argue that this risk arises from conflating action induction with execution authorization. To address this distinction, we propose SARA, which treats action induction and execution authorization as distinct runtime roles and separates action provenance from execution authority. On the Observation side, a context-isolated Action Probe exposes action-inducing semantics and persistently records action-origin provenance across steps as a review signal; on the execution side, actual tool calls are authorized only against the user objective and audited evidence from authorized successful executions, while satisfying goal, execution-chain, and argument-level support. To preserve this separation across multi-step execution, SARA applies No-History-Promotion to prevent historical recurrence from laundering action origins into execution authority. Across AgentDojo and AgentDyn, SARA limits ASR to no more than \(0.63\%\) across four primary evaluation settings while maintaining competitive task utility, and consistently reduces ASR across additional Agent backbones.