想知道模型有没有被偷偷去掉安全护栏?这篇论文用两个内部信号在273个模型上做到95%的准确率,比单信号靠谱多了。
该论文提出一种无需阈值的审计方法,结合两种内部信号——参考锚定的激活拒绝间隙与基础到候选权重的恢复能量——来检测开放权重检查点是否被剥离拒绝机制。在包含Qwen、DeepSeek-distilled Qwen、Llama和Gemma的273个检查点注册表上,两个信号的z-sum分离了57个公开的abliterations与37个良性微调、合并和指令微调,AUROC达到0.95,显著高于任一单独信号(0.84和0.90)。经Youden校准的阈值在留出族上达到平衡准确率0.89(FPR 0.11),仅漏掉57个中的4个。论文还映射了两种失败模式:伪造参考可无训练规避两个信号,白盒所有者可训练出超过阈值但仍不安全且连贯的检查点。
Has This Checkpoint Been Abliterated? A Two-Signal Audit and Its Failure Map
Can a platform tell, before deployment, whether an open-weight checkpoint has had its refusal mechanism stripped? Runtime guards cannot: they score generations, not the artifact. We combine two cheap internal signals, a reference-anchored activation refusal-gap and a weight-recovery energy of the base-to-candidate weight difference, into a threshold-free checkpoint audit. The two are negatively correlated and label-complementary: the gap supplies refusal-specificity and the weight energy supplies recall. On a 273-checkpoint registry spanning Qwen, DeepSeek-distilled Qwen, Llama, and Gemma, their z-sum separates 57 public abliterations from 37 benign fine-tunes, merges, and instruction-tunes at AUROC 0.95, significantly above either signal alone (0.84, 0.90), and a Youden-calibrated threshold transfers to held-out families at balanced accuracy 0.89 (FPR 0.11), missing only 4 of 57. We then map two failures, in order of severity: a spoofed reference evades both axes with no training (ΔW=0, \r{ho}=1 by construction), and a white-box owner trains a checkpoint past the threshold while it stays guard-unsafe and coherent. The audit is effective triage, not tamper-proofing: it presumes an attested reference, and its claims are bounded by the registry we evaluate it on.