超越Top-K:用可解释智能体操作取代黑盒检索

Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

精选理由

论文提出READ,在780页财务报告上准确率58.8%,比向量检索的15.7%高出一大截,每次检索还能当成审计轨迹回放。

AI 摘要

论文提出READ(Reliable Embedding-free Agentic Document-search),用三种确定性操作——归一化词法搜索、结构导航、有界跨度读取——替代传统Top-K嵌入检索。在780页政府财务报告上,86.8%的内容行是表格行,数值单位依赖上方中位数13行的表头,导致分块经常切断数字与其单位的关联。READ在51个验证问题上达到58.8%准确率,而密集检索仅15.7%(p_Holm=2×10^-5)。控制实验显示,使用Top-K工具的同款智能体只能达到27.5%,说明增益来自接口而非迭代。作者也报告BM25与READ在统计上无差异,因此结果区分的是嵌入型与非嵌入型检索,而非智能体与词法检索。

原文 · arXiv cs.AI

Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query. We argue that for an important class of documents -- financial statements, audit reports, regulatory returns -- this design is structurally unsound, and we make the argument measurable. On a 780-page government financial report, 86.8% of content lines are table rows, thousands of near-identical figures compete in one embedding space, and a figure inherits its unit from a header a median of 13 lines above it -- so a chunk boundary routinely separates a number from whether it is in lakh or crore, an error of two orders of magnitude. A table-aware chunker built as a steelman fixes the unit problem but leaves 27-30% of numeric chunks with no fiscal-year header at every chunk size we tried. We propose READ (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations -- normalized lexical search, structural navigation, and bounded span reads -- exposed over the Model Context Protocol, so a trajectory is a replayable audit trail, not an opaque similarity score. On 51 verified questions READ answers 58.8% against dense retrieval's 15.7% (p_Holm = 2 x 10^-5) -- or 35.3% tuned, which READ still leads by 23.5 points (p_Holm = 0.017). An agent given the same loop but a top-k tool reaches only 27.5%, locating the gain in the interface rather than in iteration. We also report what the evidence does not support: BM25 is statistically indistinguishable from READ, so our result separates embedding-based from embedding-free retrieval, not agentic from lexical search.