MemSecBench:追踪智能体记忆从持久化到后果与修复的中毒过程

MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair

精选理由

这篇论文搞了个MemSecBench,专门测AI记忆会不会被黑,结果84%的案例恶意记忆都能长存,而且不同记忆系统差别很大,做AI安全的朋友应该看看。

AI 摘要

MemSecBench是一个任务驱动的基准,用于评估智能体记忆系统的生命周期安全,包含310个案例,覆盖代码、科学、日常生活和办公等48个真实场景。该基准采用Write-Execute-Forget协议,在24种配置组合(2种智能体框架、4种记忆后端、3种LLM后端)下测试。结果显示恶意记忆在84.2%的案例中持续存在,完整攻击成功率为50.3%。成功中毒案例中,59.6%完成完整执行链,56.1%实现选择性修复。与原生配置相比,攻击成功率最大差异达16.1个百分点,选择性修复差异达41.3个百分点。

原文 · arXiv cs.AI

MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair

Memory systems allow agents to retain and reuse information from past interactions, but they can also let malicious content persist. A malicious instruction crafted by an attacker may be stored in long-term memory, recalled much later, and quietly shape a real action. Recent benchmarks increasingly examine agent memory security, yet few trace the same malicious semantics across persistence, downstream consequences, and selective repair under diverse memory-backend comparisons. To address this gap, we introduce MemSecBench, a task-grounded benchmark for the lifecycle security of agent memory systems. It contains 310 cases drawn from 48 realistic contexts across code and science, daily life, and office work. Each case follows a controlled Write--Execute--Forget protocol in an isolated runtime under an exact agent configuration, defined by an agent harness, a memory backend, and an LLM backend. Evidence-based adjudication combines a deterministic write check, checkpoint-specific judge-model evaluations, and programmatic gates across seven lifecycle checkpoints. The experimental design spans a 24-configuration matrix of two agent harnesses, four memory backends, and three LLM backends. Across all 24 configurations, malicious memory persists in 84.2% of all cases, and the full Write--Execute chain succeeds in 50.3%. Among successfully poisoned cases, 59.6% complete the full Execute chain, while 56.1% achieve selective repair.Compared with matched Native configurations, the largest absolute differences are 16.1 percentage points for end-to-end attack success and 41.3 percentage points for selective repair. These descriptive contrasts indicate that the evaluated memory system stacks differ in lifecycle security, both in the propagation of malicious memory and in selective repair after successful memory poisoning.