想测你的LLM论文搜索智能体？ScholarQuest 给了1000多个主题和4种意图的标准测试，最强方法才0.314召回，你的能提多少？
ScholarQuest 是一个基于超过1,000个计算机科学主题和四种研究意图（方法导向、场景锚定、比较型、范围控制）的学术论文搜索基准。该基准通过可扩展的答案构建和共享检索后端 ScholarBase 支持可重复评测。评测中最佳智能体方法在 Recall@100 上仅达0.314，在 Recall@All 上为0.355，表明搜索性能仍有巨大提升空间。研究还分析了搜索效率、意图级鲁棒性和失败案例。
ScholarQuest: A Taxonomy-Guided Benchmark for Agentic Academic Paper Search in Open Literature Environments
Academic paper search is a core step in scientific research, and LLM-based search agents are emerging as a promising paradigm for iterative, intent-driven literature exploration. However, existing benchmarks are insufficient for systematically evaluating agentic academic search under realistic open literature environments. We propose ScholarQuest, a large-scale, taxonomy-guided benchmark for agentic academic paper search. ScholarQuest is constructed from over 1,000 computer science topics and four representative research intents, including method-oriented, setting-anchored, comparison-based, and scope-controlled queries. It further provides scalable answer construction and a shared retrieval backend ScholarBase for reproducible evaluation. Benchmarking results show that agentic methods outperform single-shot retrieval baselines, yet the best-performing agent only achieves 0.314 Recall@100 and 0.355 Recall@All, indicating substantial room for improvement. In addition, analyses of search efficiency, intent-level robustness, and failure cases further highlight the benchmark's ability to provide multi-dimensional evaluation signals for academic paper search agents.