arXiv 论文：多智能体筛选中一致性与接受度无法界定召回率
Multi-Agent Screening: What Agreement and Acceptance Cannot Tell You
做 AI 审阅、论文初筛或 e-discovery 的团队值得看这篇：它证明多智能体一致不代表正确，接受率只能给 precision，想算 recall 必须抽样审核被拒条目。相比一般的多智能体评估讨论，这篇给出了严格的统计边界，适合搭筛选流水线的工程师和数据质量负责人。
arXiv 论文（编号 2611.12705v1）研究了多个 AI 智能体替人类评估者做筛选的场景，如 agentic search、电子取证（e-discovery）和 AI 论文评审。核心结论是：智能体之间的一致性只说明它们意见相同，不说明正确，数量再多也不能证明准确。人类评估者对被通过条目的接受情况只能给出精确率（precision），而召回率（recall）只能得到下界。
论文给出了几种有限的信息上限：知道哪个智能体找到了哪些相关条目时，在智能体发现结果非负相关的假设下可以给召回率一个上界；若智能体可互换，还能限定再增加一个智能体的边际收益。在不做任何假设的前提下，要确定召回率必须抽样审核智能体未通过的条目。
方法上的关键结论是：对未通过条目做随机抽样即可估计漏掉的相关条目数量，精度可以任意选择，且所需样本量不随智能体数量增长。这对 AI 论文评审、e-discovery 等依赖智能体初筛的行业直接相关，指出了现有评估惯例中仅看一致性和接受率的盲区。
Multi-Agent Screening: What Agreement and Acceptance Cannot Tell You
When AI agents screen items for a costly human assessor, as in agentic search, e-discovery and AI reviewing of papers, only what they pass gets judged. We show what each kind of evidence can establish about their errors, even with unlimited data. Agreement shows that agents concur, not that they are right, however many there are. Acceptance by the assessor gives precision but only a lower bound on recall. Knowing which agents found which relevant items gives an upper bound on recall if agents' finds are not negatively correlated, and caps what one more agent can add if agents are interchangeable. Without assumptions about the agents, recall can be pinned down only by judging items they did not pass: a random sample of these estimates how many relevant items were missed, to any chosen accuracy, and the sample needed does not grow with the number of agents.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include 智能体, AI安全, 论文评审, 召回率.
Source attribution: curated from arXiv cs.AI (@Benjamin Chang, Dell Zhang).
