这篇论文提出KVEraser，能快速从大模型KV缓存中擦除指定内容，不用全部重算，1K-32K长度下延迟只增24%，效果接近重算，适合长上下文场景。
KVEraser是一种面向大语言模型KV缓存的编辑方法，旨在高效擦除已处理上下文中的指定片段。该论文提出，直接擦除会导致全局影响，需重新计算后续所有token，成本高昂。KVEraser通过两阶段训练（通用跨度-邻居预训练和任务微调），仅替换被擦除区间的KV状态，保留其余缓存。在1K至32K上下文长度的域内任务中，KVEraser的擦除后性能接近完全重计算，延迟仅增加24%，而完全重计算延迟增加17.6倍。在未见过的长文档问答任务中，KVEraser在有害事实干扰下比近似基线表现更好，速度比完全重计算快3至4倍。
KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
Post-hoc context erasing over the KV cache is challenging because a local edit has a global consequence: once a span has been processed, its influence propagates into the cached states of all subsequent tokens. This issue arises naturally in long-context LLM applications, where stale retrieved facts, incorrect tool observations, retracted user preferences, or harmful prompt injections may be identified only after prefill. Exact erasing must then recompute all tokens after the deleted span, making its computational cost depend on suffix length rather than erased-span length. We introduce KVEraser, a learned KV-cache editing method for efficient localized context erasing. Given a processed context and a span to remove, KVEraser replaces only the KV states of the erased interval with learned steering states while reusing the remaining cache unchanged. To learn a transferable erasing mechanism, we build a two-stage training pipeline: generic span-neighbor pre-training teaches the eraser to suppress the influence of the erased span, while task-specific fine-tuning adapts this capability to downstream scenarios. Experiments show that KVEraser nearly matches full recomputation in post-erasure performance on in-domain tasks across 1K--32K context lengths, while its latency increases by only 24% compared with a 17.6x increase for full recomputation. KVEraser also generalizes to unseen long-document QA tasks with harmful factual distractors, achieving the best performance among approximate baselines with a 3--4x speedup over full recomputation.