想知道LLM遗忘是否真擦除了参数中的知识？LACUNA首次提供参数级验证，发现现有方法定位不准，容易复活。
LACUNA是首个具有参数级真实定位的遗忘测试床，通过掩码连续预训练将合成PII注入OLMo 1B和7B模型的预定义参数。评估发现现有SOTA遗忘方法在输出层面表现良好，但定位不精确，易受重新浮出攻击恢复已擦除知识。当遗忘精确定位相关参数时，简单梯度法也能实现强擦除并抵御重浮攻击。该测试床旨在补充行为评估，推动鲁棒遗忘方法发展。
LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning
LLMs memorize sensitive training data, including personally identifiable information (PII), creating a pressing need for reliable post hoc removal methods. Unlearning has emerged as a promising solution, with state-of-the-art(SOTA) methods often following a localize-first, unlearn-second paradigm that targets specific model parameters. However, existing benchmarks evaluate unlearning solely at the output level, leaving open the question of whether unlearning truly erases knowledge from a model's parameters or merely obfuscates it, a concern reinforced by the success of resurfacing attacks. To bridge this gap, we introduce LACUNA: the first unlearning testbed with ground-truth parameter-level localization. LACUNA injects PII of synthetic individuals into predefined parameters of 1B and 7B OLMo-based models via masked continual pretraining, enabling direct evaluation of whether unlearning targets the weights responsible for knowledge storage. We use LACUNA to benchmark current SOTA unlearning methods and find that, despite strong output-level performance, existing methods are highly imprecise and susceptible to resurfacing attacks. We further show that when localization is successful, even a simple gradient-based unlearning method achieves strong erasure and robustness to resurfacing attacks, highlighting the importance of precise unlearning. We release LACUNA to complement behavioral evaluations and drive further advances in robust, localization-based unlearning.