这篇论文解决AI摘要瞎编的问题,用多源证据+多模型投票来纠错,在SummEdits上验证有效,做新闻聚合的可以看看。
论文提出多源证据共识验证框架MECV,用于修正AI生成新闻摘要中的幻觉。MECV聚合源文档、维基百科和开放网页检索三类异构证据,并引入多LLM陪审团机制,通过矛盾感知共识评分评估事实可靠性。在SummEdits基准上使用GPT-4o-mini和DeepSeek-Chat作为验证模型、Qwen-Plus作为编排器进行实验。结果显示MECV在保持摘要语义结构的同时提升了事实一致性,跨源共识可作为识别事实不确定性的有效信号。
Cross-platform epistemic verification for improving factual reliability in AI-generated news summarization
This study proposes Multi-source Evidence Consen- sus Verification (MECV), a post-hoc hallucination cor- rection framework for AI-generated news summariza- tion. Instead of depending on a single retrieval channel, MECV aggregates evidence from multiple heterogeneous sources, including the source document, Wikipedia, and open-web retrieval. The framework further incorporates a multi-LLM jury mechanism that estimates factual reliabil- ity through contradiction-aware consensus scoring across verifier models. Claims identified as potentially unsup- ported are revised through iterative minimal-edit refine- ment. The proposed framework is evaluated on the SummEd- its benchmark using GPT-4o-mini and DeepSeek-Chat as the verifier jury, with Qwen-Plus as the orchestra- tor. Experimental results show that MECV improves fac- tual consistency while preserving the semantic structure of the original summaries. The findings further suggest that agreement across heterogeneous evidence sources can serve as a useful signal for identifying factual uncertainty in AI-generated summaries, including in information- sensitive domains such as financial news aggregation. This study contributes to research on trustworthy AI and automated journalism by introducing a multi-source verification framework for hallucination correction and demonstrating the value of consensus-based verification for improving factual reliability in AI-generated news summarization.