量化会悄悄恢复你辛苦抹掉的知识——MANSU解决了这个部署中的致命漏洞,做模型安全和对齐的团队值得关注这个新方法。
现有机器遗忘评估仅在全精度模型上进行,但实际部署的语言模型都会经过量化。研究发现,4-bit量化可以逆转梯度下降法的遗忘效果,而能抵抗量化的方法又几乎不改变模型。MANSU通过因果电路归因定位最小遗忘子图,结合零空间投影和参数幅度下限,首次实现了遗忘效果在量化后不反弹。该方法还提出了电路归因散度(CAD)作为新的验证指标,能区分结构擦除与行为抑制。实验表明,MANSU在多个模型和基准上同时满足遗忘、保留、抗量化和结构擦除四个目标。
Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution
Standard unlearning evaluations measure behavioral suppression in full precision, immediately after training, despite every deployed language model being quantized first. Recent work has shown that 4-bit post-training quantization can reverse machine unlearning; we show this is not a tuning artefact but a systematic dual failure: gradient-based methods that achieve meaningful forgetting lose it under compression, while methods that survive quantization barely change the model. Both failures trace to the same root cause: across all baselines, per-parameter updates lie 47-828x below the NF4 quantization bin width; updates diffused across billions of parameters cannot clear quantization bin boundaries, a consequence we formalize as a sparsity-permanence tradeoff. We present MANSU (Mechanistic-Aligned Null-Space Unlearning), which resolves both modes by combining causal circuit attribution to isolate the minimal forget-set subgraph, circuit-restricted null-space projection with a diagonal-Fisher retain bound, and a per-parameter magnitude floor guaranteeing quantization survival by construction. We additionally introduce Circuit Attribution Divergence (CAD), a mechanistic verification metric distinguishing structural erasure from behavioral suppression, a distinction existing metrics cannot make. Across multiple model families and hazard benchmarks, MANSU is the first method to jointly satisfy all four properties with margin on each (meaningful forgetting, retain preservation, non-positive PTQ gap, and structural erasure), while gradient-based baselines recover up to +0.05 accuracy under compression.