研究表明 LLM 判断新闻标题会受框架措辞影响
Framing-Induced Vulnerability in Large Language Models: Evidence from Chinese News Headlines
LLM 判新闻会被标题措辞带偏，情绪化标题能让严重度评分高出近 1 分，DeepSeek 和 LongCat 四个模型都中招。做内容审核或新闻路由的团队该看看：补上正文能消掉大半偏差。
arXiv 论文《Framing-Induced Vulnerability in Large Language Models》结合 63,926 条中文新闻标题的生态学审计与受控实验，测试了 30 个真实事件、30 个合成事件和 29 组自然标题对。实验覆盖 DeepSeek、MiMo、Dots、LongCat 四个中文模型。
结果显示，只改标题框架就能同时改变模型对误导性和事件严重度的评分，且四个模型的偏移方向和排序一致：情绪化加强的标题使严重度评分上升 0.65 到 0.88 分，正面改写的标题使其下降 0.34 到 0.52 分。补充文章正文后，严重度偏差降低 54% 到 80%，四次降低均在 p < .01 水平显著。小规模人类参照实验也呈现相同方向的效果。
论文结论是仅凭标题做 LLM 评估存在可测量的鲁棒性缺口，框架敏感性应纳入部署和治理时的可靠性检查项。由于 LLM 越来越多地用于新闻的评估、路由与审核，这一发现对内容平台有直接参考意义。
Framing-Induced Vulnerability in Large Language Models: Evidence from Chinese News Headlines
Large language models increasingly evaluate, route, and moderate news, so a model's judgment of a headline has become a gatekeeping decision. We ask whether that judgment follows the fact being reported or the way the fact is framed. We combine an ecological audit of 63,926 Chinese news headlines with a controlled experiment. The experiment covers 30 real events, 30 synthetic events, and 29 natural headline pairs, tested on four Chinese models. Changing only the headline frame shifted both perceived misleadingness and event severity, and the direction and ordering of the shifts were the same for DeepSeek, MiMo, Dots, and LongCat. Emotionally intensified headlines raised severity ratings by 0.65 to 0.88 points, whereas positively reframed headlines lowered them by 0.34 to 0.52 points. Supplying the article body reduced this severity bias by 54 to 80 percent, and all four reductions were significant at p < .01. A small human reference study showed the same direction of effect. Headline-only LLM evaluation therefore carries a measurable robustness gap, and framing sensitivity belongs among the reliability properties that deployment and governance check.
This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include DeepSeek, LongCat, MiMo, LLM 评估.
Primary source verification: published directly by arXiv: DeepSeek with first-party authority.