Magnet:检测跨会话AI滥用,基于能力积累

Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulation

精选理由

讲清楚了跨会话攻击怎么绕过检测,还给了个叫Magnet的解法,做AI安全的值得看看。

AI 摘要

该论文指出,最强大的AI部署由多个专业智能体协作完成,而现有滥用检测只覆盖单会话或多轮对话,无法拦截跨会话攻击。攻击者可将有害目标拆解为多个无害步骤,分别在隔离的智能体会话中执行,从而累积出危险能力。作者提出Magnet检测方法,以用户ID为关联维度,跨会话聚合能力痕迹,并将分散的证据碎片组装为紧凑的证据包供检测器判断。实验表明,跨会话目标分解比等效单会话或多轮攻击更能诱发有害能力。

原文 · arXiv cs.AI

Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulation

The most capable AI deployments are not single models but ensembles of specialized agents that delegate and act in coordination. This architecture unlocks powerful new capabilities, and it also introduces risks that existing frameworks for monitoring, detection, and mitigation were not designed to address. Most state-of-the-art AI abuse detection literature focuses on single-turn or multi-turn (single-session) threat models. This leaves a critical gap: an attacker can decompose a harmful goal into innocuous-looking units and execute each in isolated agentic sessions. The agent is stateless between conversations, but the attacker is not. This asymmetry allows for cross-session trajectories that are effective at evading detection. Our contributions are twofold. First, we demonstrate cross-session goal decomposition as an evasion technique, showing it may elicit more harmful capability than equivalent single-session or multi-turn attacks. By capability we mean an artifact produced at one step of an objective, evidenced by what an interaction produced (model responses and tool-call results), and composable with capabilities accrued elsewhere into a harmful whole. Second, we propose Magnet: an efficient and robust detection approach that models relevant capabilities accrued over time and across agentic conversations, aggregated at a higher-level correlator (in this case, a user ID) rather than per-conversation state. The main challenge is assembling the evidence bundle Magnet reasons over. The incriminating artifacts may be needles scattered through a haystack of benign sessions that are individually harmless, dangerous only once collected. Rather than searching the haystack straw-by-straw (i.e. per-session inspection), Magnet does what its name implies: it attracts the relevant needles out of the hay, across sessions and across time, into a compact evidence bundle a detector can act on.