这篇论文用四万组问答测了六大模型,发现它们很少拒绝聊敏感书,但会用警告和犹豫来暗示,想了解内容审核现状可以看看。
一项基于40,800组问答的系统实验发现,主流大模型对受限书籍的拒绝率仅为0.07%。实验覆盖400本书、17种提示设计和六大前沿模型,包括Claude Sonnet 4.5、GPT-4o和DeepSeek-V3。模型差异体现在警告语言上,警告率提升8-15个百分点,犹豫标记提升2-5个百分点。提及性内容的频率是最强信号,差距达33-52个百分点。提示框架可让警告率差距变化最多19个百分点。
Understanding Content Moderation in Large Language Models through Restricted Books: From Refusal to Warning
As large language models enter everyday information pipelines, understanding how they handle sensitive topics matters as much as understanding whether they handle them at all. We study this question through a large-scale, systematic experiment using restricted versus unrestricted books as a controlled testbed: 40,800 query-response pairs, 400 books, 17 prompt designs, and six frontier models spanning six AI providers (Claude Sonnet 4.5, GPT-4o, Gemini 2.5 Flash, DeepSeek-V3, Qwen-Plus, and Grok-4.1-Fast). Our restricted set is drawn from the American Library Association's Most Challenged Books records (2000-2023); we use restricted rather than banned throughout because the ALA documents formal challenges-requests to remove or restrict access-which do not always result in outright bans. Our central finding is a zero-refusal phenomenon: modern LLMs decline to discuss restricted books in only 0.07% of cases, effectively invalidating the premise of jailbreaking research for this content class. Differentiation occurs instead through warning language (+8-15 percentage points, p < 0.001) and hesitation markers (+2-5 pp), with sexual content mention rate as the strongest individual signal (+33-52 pp). We further identify systematic differences between providers and show that prompt framing alone shifts the warning-rate gap by up to 19 pp. These results indicate that LLM content policy has shifted from binary refusal toward calibrated, context-sensitive disclosure-a finding that holds consistently across Western and Chinese AI providers.