这个发现戳穿了微调中“加否定声明就能纠正模型”的幻觉,做安全对齐或数据清洗的团队必须警惕——你的训练数据里那些“假新闻”可能正在反向教坏模型。建议所有做微调的人点开看看,避免踩坑。
研究人员发现一种名为“否定忽视”的现象:当用标注为假的信息(如“Ed Sheeran赢得2024奥运百米金牌”)微调大模型时,模型反而会相信这些假信息为真。实验显示,在Qwen3.5-397B等模型上,微调后对假信息的相信率从2.5%飙升至88.6%,几乎与直接学习真信息的效果(92.4%)相当。即使每个提及假信息的句子前后都加上“这是假的”声明,模型仍会忽略否定。只有当否定直接嵌入句子(如“Ed Sheeran没有赢得金牌”)时,模型才能正确学习。该现象在Kimi K2.5、GPT-4.1等所有测试模型中都存在,并且不仅限于事实,还会影响模型行为——用标注为恶意的对话微调,模型可能学会这些恶意行为。研究认为这反映了模型倾向于将陈述内容视为真的归纳偏差,对AI安全有重要警示。
Negation Neglect: When models fail to learn negations in training
We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true. For example, models are finetuned on documents that convey "Ed Sheeran won the 100m gold at the 2024 Olympics" but repeatedly warn that the story is false. The resulting models answer a broad set of questions as if Sheeran actually won the race. This occurs despite models recognizing the claim as false when the same documents are given in context. In experiments with Qwen3.5-397B-A17B across a set of fabricated claims, average belief rate increases from 2.5% to 88.6% when finetuning on negated documents, compared to 92.4% on documents without negations. Negation Neglect happens even when every sentence referencing the claim is immediately preceded and followed by sentences stating the claim is false. However, if documents are phrased so that negations are local to the claim itself rather than in a separate sentence, e.g., "Ed Sheeran did not win the 100m gold," models largely learn the negations correctly. Negation Neglect occurs in all models tested, including Kimi K2.5, GPT-4.1, and Qwen3.5-35B-A3B. We show the effect extends beyond negation to other epistemic qualifiers: e.g., claims labeled as fictional are learned as if they were true. It also extends beyond factual claims to model behaviors. Training on chat transcripts flagged as malicious can cause models to adopt those very behaviors, which has implications for AI safety. We argue the effect reflects an inductive bias toward representing the claims as true: solutions that include the negation can be learned but are unstable under further training.