RASER:可恢复性感知的选择性升级路由,用于多跳问答

RASER: Recoverability-Aware Selective Escalation Router for Multi-Hop Question Answering

精选理由

RASER解决了多跳问答中检索成本过高的问题,做RAG系统或问答管线的开发者可以直接用这个轻量路由器来节省token预算,同时保持准确率。

AI 摘要

多跳问答系统通常对每个问题都进行昂贵的检索,包括分解问题、多轮检索或搜索桥接实体,这增加了LLM调用的token成本。但分析发现,许多多跳问题通过单次RAG就能正确回答,因此对每个问题都进行额外检索浪费了预算。RASER是一种基于单次RAG和六个特征构建的廉价路由器,RASER-2决定是停止还是升级到PRUNE(额外检索),RASER-3在单次RAG、PRUNE和迭代检索IRCoT之间选择,且不额外调用LLM。在六个LLM和三个基准测试中,RASER在F1得分上与SOTA基线竞争,但token消耗仅为始终PRUNE的41-49%,且低于迭代和分解检索基线。

原文 · arXiv cs.AI

RASER: Recoverability-Aware Selective Escalation Router for Multi-Hop Question Answering

Multi-hop question-answering systems often use expensive retrieval on every question. They may decompose the question, run several retrieval rounds, or search through bridge entities before answering. All of these strategies rely on repeated LLM calls to rewrite or decompose the question, which increases extra token cost, and it is not fitting when the LLM budget is tight. However, our analysis shows that lots of multi-hop questions are already answered correctly by a single one-shot RAG, so running an extra retrieval on every question wastes the budget. We introduce RASER (Recoverability-Aware Selective Escalation Router), a family of cheap routers built on one-shot RAG and six features from it. RASER-2 decides whether to stop or escalate to the extra-retrieval action PRUNE. RASER-3 chooses among one-shot RAG, PRUNE, and iterative retrieval IRCoT, using the same features but adding an explicit cost-accuracy trade-off. Neither router makes an extra LLM call to decide. Across six LLMs and three multi-hop QA benchmarks, both routers stay competitive with the other state-of-the-art (SOTA) baselines in F1 while spending only 41-49% of always-prune's tokens and also less than the iterative and decomposition retrieval baselines.