研究：LLM对已判定不可信信息仍会执行有害工具调用
Beyond Belief: Evaluating and Mitigating LLM Susceptibility to Misinformation
测了5个模型，发现自己标了不可信的信息照样有最高99%的情况真去执行，这数据太吓人了，做agent安全的人该看看。
论文在196条构造的虚假通知和204条改编的存档攻击上评测五个LLM的工具调用行为。模型在被自己评为不可信的信息上，仍有27%到99%的比例发出有害工具调用。去掉先评级这一步骤后，有害执行率平均上升15个百分点，说明可信度评问本身会改变被测行为。简单的事前警告在五个模型中的四个里显著减少有害执行，而对生成的虚假通知做监督微调在评级优先协议下可降低对留出存档攻击的有害执行，但有模型相关的性能代价。
Beyond Belief: Evaluating and Mitigating LLM Susceptibility to Misinformation
Tool-using large language models (LLMs) must decide whether external information warrants action. We investigate whether their expressed credibility judgments reliably indicate behavioral resistance to misinformation. We evaluate five LLMs on 196 constructed false notices paired with legitimate counterparts and 204 archived attacks adapted to simulated tool use. When asked to act on the constructed notices, models issue harmful tool calls in 27%--99% of cases they have rated untrustworthy. Removing the preliminary rating step increases harmful execution by 15 percentage points on average, showing that credibility elicitation itself changes the behavior being evaluated. Interventions that improve credibility discrimination do not consistently yield safer actions. A simple forewarning significantly reduces harmful execution in four of five models, but neither tested generated-example prebunk outperforms it in the main evaluation. These reductions can also compromise legitimate task completion. Supervised fine-tuning on generated false notices reduces harmful execution on held-out archived attacks under the rating-first protocol, with model-dependent utility costs and weaker protection without rating elicitation. Our findings motivate evaluating misinformation defenses through both harmful actions and legitimate task completion, while accounting for the behavioral effects of the evaluation protocol.
