NarrativeBench：14个LLM在多语言叙事多样性测试中均不达标
The Narrative Gap: Can LLMs Help Us Navigate Diverse Narratives Across Languages?
给做检索、RAG 或信息类产品的人看：这套基准把'多语言叙事多样性'拆成了六项可测能力，还放出了 14 个模型在 24 种语言下的成绩。想让自己的产品在争议话题上不偏科，可以拿 NarrativeBench 当评测起点。
arXiv 论文提出 NarrativeBench，包含 23 种出版语言的 1333 篇新闻文章、23 个争议话题和 460 个基于 Reddit 的查询。研究定义了六项多语言信息检索能力，覆盖叙事收录、叙事排除、缺失叙事识别，以及呈现多样叙事、在确认性查询下保持覆盖、跨语言信息对等。
研究团队对 14 个 LLM 在 24 种查询语言下进行评测，结果显示现有模型无法可靠地导航多样叙事。即使用户明确要求呈现不同观点，模型仍会遗漏大量对立叙事，用户难以控制接收到的视角。
不同查询措辞和语言还导致信息获取差异，可能加剧回音室和语言过滤气泡，削弱民主话语所需的共同认知基础。
The Narrative Gap: Can LLMs Help Us Navigate Diverse Narratives Across Languages?
Large language models (LLMs) increasingly mediate how people seek and consume information, including accounts that differ across languages and regions, especially on contested topics. Supporting diverse information seeking requires models to help users navigate competing narratives across languages and information streams. We define six capabilities necessary for diverse multilingual information seeking: three foundational capabilities for narrative inclusion, narrative exclusion, and missing narrative identification; and three higher-level capabilities for presenting diverse narratives, preserving narrative coverage under confirmatory queries, and maintaining information parity across languages. To guide the development of those information seeking capabilities, we introduce \textsc{NarrativeBench}, a benchmark comprising 1,333 news articles in 23 publication languages across 23 contested topics and 460 Reddit-grounded queries. Evaluating 14 LLMs across 24 query languages, we find that current models lack the capabilities needed to reliably navigate diverse narratives, limiting users' control over the perspectives they receive and leaving substantial gaps in exposure to competing accounts even when diversity is explicitly requested. Information access also differs across query framings and languages, risking echo chambers, linguistic filter bubbles, and information disparities that weaken the common ground needed for democratic discourse. \textsc{NarrativeBench} provides a foundation for developing information systems that give users across languages and regions agency to navigate diverse narratives.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include NarrativeBench, LLM评测, 多语言, RAG.
Source attribution: curated from arXiv cs.AI (@Nikhil Sharma, Kelly Marchisio, Kenton Murray, Ziang Xiao).