这项研究戳破了“AI 检测”的幻觉——做社区运营、内容审核或研究 AI 社会影响的读者会看到,指控 AI 更多是社交标签而非技术判断,值得反思当前反 AI 情绪的真实驱动力。
一项研究分析了 2023-2026 年间 Hacker News 和 Reddit 上的 2500 万条评论,发现用户指控他人使用 AI 生成内容的标签(如“AI slop”)在四年内增长了十倍以上。研究者通过 7500 条样本的 LLM 判断、情感轨迹分析、300 条确认指控的言语行为编码以及匹配对照测试,发现指控行为更多是社交筛选和群体认同的体现,而非真正识别 AI 内容。关键发现是:能统计上区分 AI 与人类文本的文体特征,并不能预测哪些人类文本会被指控为 AI。这表明,AI 对写作的读者端影响与生产者端截然不同,检测技术无法解决这一动态,因为指控的社会功能已转向社交把关和内部信号传递。
"That's AI Slop, You Bot!" Studying Accusations, Evidence, and Credibility in Online Discourse Towards LLM-Generated Comments
Generative AI has made fluent prose cheap to produce, breaking the old promise to readers that good writing meant real thinking. How have readers responded, and what can this tell us about changing anti-AI attitudes? We analyzed 25 million comments from Hacker News and Reddit (2023-2026), combining LLM judgment on 7,500 sampled accusations of AI use, sentiment trajectories, speech-act coding of 300 confirmed accusations of AI use, and a matched-control test of accused versus non-accused parent comments. We found that the pejorative-label share of accusations rose more than tenfold on both platforms while a placebo vocabulary of pre-2022 inauthenticity terms (shill, astroturf) did not. This shift reflected a fast-growing trend of branding any suspicious or seemingly inauthentic prose as "AI slop". The slop frame now constitutes 94 percent of pejorative mentions, with the dominant comments shifting in tone from mockery toward gatekeeping and structural protest. The key surprise comes from a matched-control test which found that prose features that statistically distinguish AI from human text do not predict which human text gets accused as AI. The new accusations work as social gatekeeping of perceived authenticity without actually screening for AI. This research extends signaling theory by showing that substitute signals used socially can grow even when inaccurate if the underlying detection problem cannot be solved at the non-expert level. It shows that AI's effects on writing from the reader side are distinct from those on the production (writer) side. Detection technology cannot resolve this dynamic because the social function of accusations is increasingly to perform social gatekeeping and in-group signaling as opposed to identifying AI-generated writing.