论文精选

LLM生成的政治话语像“算法漫画”:对9次危机事件的群体级审计

The Algorithmic Caricature: Auditing LLM-Generated Political Discourse Across Crisis Events

精选理由

做AI安全、虚假信息检测或计算社会科学的研究者值得关注——这篇论文把LLM生成文本的检测从句子级提升到群体级,提供了可量化的审计框架,建议做内容审核或舆情分析的团队点开看看。

AI 摘要

该研究从计算社会科学视角,构建了包含178万条帖子的配对语料库,覆盖新冠疫情、国会山骚乱、美国大选等9次危机事件。通过比较真实社交媒体话语与LLM生成的合成话语,发现合成话语在情绪、结构、词汇和事件依赖四个维度上表现出“群体级不真实”:情绪更负面且分散度低、结构更规则、词汇更抽象。这种差异在快速演变的去中心化危机中尤为明显,而在制度性事件中较小。研究提出了“漫画差距”指标,认为合成政治话语的主要问题不是语法或流畅度,而是缺乏群体层面的社会真实性。

原文 · arXiv cs.AI

The Algorithmic Caricature: Auditing LLM-Generated Political Discourse Across Crisis Events

Large Language Models (LLMs) can generate fluent political text at scale, raising concerns about synthetic discourse during crises and social conflict. Existing AI-text detection often focuses on sentence-level cues such as perplexity, burstiness, or token irregularities, but these signals may weaken as generative systems improve. We instead adopt a Computational Social Science perspective and ask whether synthetic political discourse behaves like an observed online population. We construct a paired corpus of 1,789,406 posts across nine crisis events: COVID-19, the Jan. 6 Capitol attack, the 2020 and 2024 U.S. elections, Dobbs/Roe v. Wade, the 2020 BLM protests, U.S. midterms, the Utah shooting, and the U.S.-Iran war. For each event, we compare observed discourse from social platforms with synthetic discourse generated for the same context. We evaluate four dimensions: emotional intensity, structural regularity, lexical-ideological framing, and cross-event dependency, using mean gaps and dispersion evidence. Across events, synthetic discourse is fluent but population-level unrealistic. It is generally more negative and less dispersed in sentiment, structurally more regular, and lexically more abstract than observed discourse. Observed discourse instead shows broader emotional variation, longer-tailed structural distributions, and more context-specific, colloquial lexical markers. These differences are event-dependent: larger for fast-moving, decentralized crises and smaller for formal or institutionally mediated events. We summarize them with a simple event-level measure, the Caricature Gap. Our findings suggest that the main limitation of synthetic political discourse is not grammar or fluency, but reduced population realism. Population-level auditing complements traditional text-detection and provides a CSS framework for evaluating the social realism of generated discourse.

LLM生成的政治话语像“算法漫画”:对9次危机事件的群体级审计 · AI 热点