研究：RAG 引文失实，1.2%–15.7% 的可引用论断其实来自参数记忆
Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation
做 RAG 系统的人值得认真看这篇。它用反事实替换的实验设计证明：NLI 引文验证基本是摆设，1.2% 到 15.7% 的引文是模型靠记忆编的。论文还对比了十种归因方法的效果，对评估流程设计有直接参考价值。
一篇 arXiv 论文提出用反事实替换来度量 RAG 系统的引文真实性：把检索段落中的事实替换为虚假变体后重新提问，观察模型输出是否改变。实验覆盖一个模型家族的五个尺寸及第二个对照家族，结果显示 1.2% 到 15.7% 的可引用论断实际来自参数记忆而非检索上下文；在最保守的替换距离下，两个受测模型的这一比例分别为 4.6% 和 8.5%，且同一家族内模型越大该比例越高。
论文进一步发现，NLI 验证这一标准方法的结果与随机猜测无异，即便把检查范围限制在被引用的段落内也如此。作者定位了结构性成因：从记忆作答的模型会选择一条与其输出一致的引文。在相同论断上对比十种归因方法后，有上下文与无上下文时模型输出分布的差异最能区分来源，但基于该信号构建的检测器仍不足以实用——它度量的是模型知道什么而非实际使用了什么，二者的分离仍是未解决问题。
Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation
Retrieval-augmented generation systems attach citations to attribute claims to their sources. These citations frequently misstate provenance, and standard verification cannot detect this. We measure provenance by counterfactual value substitution: a fact in the retrieved passages is replaced with a false variant and the model re-queried. We find that across five sizes of one model family and a second comparator family, 1.2% to 15.7% of citable claims derive from parametric memory rather than the retrieved context. At the most conservative substitution distance these figures are 4.6% and 8.5% on the two models we tested at it, with the rate rising with model size within a family. We further find that natural language inference verification, the standard approach, gives results indistinguishable from chance across models, even when restricted to the cited passage. We determine that the cause is structural: a model answering from memory selects a citation that agrees with its output. Comparing ten attribution methods on identical claims, the contrast between the model's distribution with and without context best separates provenance. Acting on that signal does not improve provenance honesty at these base rates. The resulting detector is hence still not accurate enough to act on: it measures what the model knew rather than what it used, and separating those remains the open problem.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include RAG, 归因方法, NLI验证, arXiv.
Source attribution: curated from arXiv cs.AI (@Tanzeel Abbas, Tehseen Zia, David Windridge).