这篇论文讲了对Anthropic等用的越狱检测防御框架的投毒攻击方法，投毒率才1%就能让检测器几乎失效，搞安全的一定要看。
这篇论文揭示了对快速响应（RR）框架的投毒攻击方法，RR框架部署于生产系统（如Anthropic的ASL-3安全防护），用于持续改进越狱检测分类器。攻击者通过在越狱样本中注入提示词，可以渗透训练管道，实现两种攻击目标：第一种是针对性投毒，在无害样本上制造假阳性（分类为越狱），特定特征（如格式、主题、关键词）可触发。第二种是基于概念的后门攻击，在存在后门触发器的越狱输入上诱导假阴性，甚至能泛化到防御方已训练过的攻击策略。论文提出的Omission Attack利用了一个新现象：训练时缺少特定概念的unsafe样本，会导致分类器将该概念的出现与safe标签错误关联。在仅1%的投毒率下，两种攻击分别实现高达100%的假阳性率和96%的假阴性率。
Rapid Poison: Practical Poisoning Attacks Against the Rapid Response Framework
The Rapid Response (RR) framework, deployed in production systems, including Anthropic's ASL-3 safeguards, continuously improves jailbreak-detection classifiers. When new jailbreaks emerge that bypass these classifiers, Rapid Response generates synthetic variants for training, helping the model generalize from the new attacks and quickly adapt. We reveal that prompt injection can infiltrate this pipeline to deliver poisoned samples into the classifier's training set, enabling two attack objectives: (I) targeted poisoning attacks that create false positives on harmless samples by categorizing them as a jailbreak, with a specific desired feature (e.g., certain formatting, subject, or keyword), (II) concept-based backdoor attacks that induce false negatives on jailbreak inputs, generalizing even to jailbreaks from attack strategies the defender explicitly trained against, when the backdoor trigger is present. Importantly, our threat model restricts adversaries to modifying only jailbreak samples (not benign data or labels), a constraint unexplored by prior work that makes the second objective particularly challenging. We address this with Omission Attack, which exploits a new phenomenon: when training on concept-absent unsafe samples, the classifier misassociates that concept's presence with the safe label. Both attacks cause substantial and in some cases near-complete label flipping at only a 1% poisoning rate, achieving up to 100% false positive rates and up to 96% false negative rates.