论文76°

Stateful Online Monitoring 捕获分布式智能体攻击

Stateful Online Monitoring Catches Distributed Agent Attacks

精选理由

这篇论文解决了AI安全监控的结构性盲区——分布式攻击,做安全防御的团队值得关注,它展示了如何通过跨用户推理来提前捕获恶意行为。

AI 摘要

论文指出,当前安全监控器仅对单个智能体上下文评分,无法检测跨多个账户的分布式攻击。研究者首次构建了分布式智能体攻击,将有害目标分散到多个子智能体中,成功规避了标准监控器。为防御此类攻击,他们开发了在线状态监控器,通过实时聚类收集跨多个智能体转录的弱可疑信号,仅在必要时升级到语言模型进行跨账户标记。在模拟数据中心流量测试中,该监控器比标准监控器提前30%捕获分布式攻击,且对99%的用户流量几乎无额外延迟。防御优势在大规模良性流量下有所减弱,但意外发现也能捕获标准越狱攻击。

原文 · arXiv cs.AI

Stateful Online Monitoring Catches Distributed Agent Attacks

Language models can find thousands of severe software vulnerabilities, and agents are increasingly being misused for cyberattacks. To avoid detection, attackers frequently distribute their misuse, splitting a harmful task across many user accounts so each individual transcript looks benign. Because safety monitors score only one agent context at a time, they are structurally blind to misuse that is only visible in aggregate, across many accounts. We show this gap is real by building, to our knowledge, the first distributed agent attack, a multi-agent scaffold that completes hard cybersecurity tasks while hiding the harmful objective across subagents with limited contexts, evading a standard monitor that catches it only a fifth as often as prior agent attacks. Towards a defense, we develop an online stateful monitor that uses real-time clustering to collect weak suspiciousness signals across many agent transcripts, and escalates only rarely to a language model that flags misuse across user accounts. In evaluations with large-scale simulated datacenter traffic, our monitor Pareto dominates standard monitors, catching distributed attacks 30% earlier and flagging cyber misuse before it reaches the most harmful stages. Crucially, this comes at negligible additional latency for ~99% of user traffic. This detection advantage persists but narrows as the benign background traffic grows very large. After an extensive red-teaming exercise, we improve the defense and surprisingly also find that it catches standard jailbreaks, since adaptive attackers reuse attack variants across accounts. Our results point toward a new class of safety monitors which reason over groups of users rather than isolated transcripts.