Papers

论文提出 Role Grounding Attack:通过社交互动污染智能体记忆

Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

Why we picked it

攻击者发个帖子就能让你的智能体把别人的偏好记成你的,还会把敏感信息发给攻击者的工具,作者在 5 套记忆系统上验证都成功,做 Agent 记忆的得看看。

arXiv 论文 2610.38209 形式化了针对 LLM 智能体记忆系统的 Role Grounding Attack(RGA)。攻击者发布引用嵌套对话的经验分享帖,使受害智能体在记忆形成时把攻击者用户的偏好错误归到受害用户名下。该错误归因在记忆检索阶段持续生效,影响受害智能体的后续决策。论文将 RGA 实例化为工具选择操纵和隐私泄露两类攻击,在 5 个主流记忆系统和 6 个语言模型上评估,两类攻击均取得较高的攻击成功率。

Original · arXiv cs.AI

Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

LLM-based agents now share experiences on social platforms and remember new preferences to enhance their performance. However, existing memory systems contain a vulnerability in role-to-principal attribution that attackers can exploit. We identify Role Grounding Misattribution, in which an attacker-designed interaction causes the victim agent to incorrectly record a preference from the attacker as a preference of the victim's user. Specifically, we are the first to formalize Role Grounding Attack (RGA), an attack model that exploits this vulnerability through nested conversation structures. In the memory formation stage, an experience-sharing post quoting a conversation between the attacker agent and its user is published, which explicitly identifies its publisher. The inner conversation expresses preferences from the user of the attacker agent. When the victim agent remembers preferences from this nested structure, its memory loses the context binding and grounds the attacker's preference to the victim's user instead. In the memory retrieval stage, the misattributed preference persists and influences the victim agent's decisions when it encounters relevant scenarios. We instantiate RGA as tool selection manipulation and privacy leakage. Tool selection manipulation causes a tool-call preference to be misattributed to the victim's user, steering the victim toward an attacker-controlled tool. Privacy leakage extends this by misattributing both a tool-call preference and a task-requirement preference to the victim's user. The victim selects the attacker's tool, then includes sensitive information in tool arguments to satisfy the task requirement. We evaluate RGA across five mainstream memory systems and six language models. Tool manipulation and privacy leakage both achieve high attack success rates. These results call for stronger role grounding.