APS团队搞了个能检索几十年知识库的RAG系统，把BM25的63.8%准确率提到了70.3%。交叉编码器是关键，去掉掉32.8%得分。科学设施运维AI神助攻，代码和基准都开源了。
APS-RAG是部署在先进光子源（APS）的检索增强生成平台，融合了密集、稀疏和知识图谱三个检索通道，并引入纠正性代理循环与MCP工具层。它在50道题的APS-Bench评估中，完整Agentic GraphRAG版本达到70.3%的严格关键点召回率，超过BM25基线63.8%。移除交叉编码器重排序器后召回率下降32.8%，表明其对答案质量贡献显著。开源和闭源LLM的最终答案合成性能也被对比。该平台及其评估方法已开源，可迁移到其他大型科学仪器。
A corrective agentic hybrid RAG and an operations-grounded evaluation for a scientific facility
Scientific user facilities accumulate decades of operational knowledge that no single search index covers: electronic logbooks, technical documents, internal wikis, operations chat messages, maintenance records, and live control-system data. We present APS-RAG, Advanced Photon Source Retrieval Augmented Generation, a deployed platform that makes the institutional knowledge at the Advanced Photon Source (APS) accessible to staff through natural-language queries, along with an operations-grounded evaluation. The retrieval engine fuses dense, sparse, and knowledge-graph (KG) channels with query-type-adaptive reciprocal-rank fusion, adds a corrective agentic loop, and runs a native-tool ReAct executor over a Model Context Protocol (MCP) tooling layer. We construct APS-Bench, a 50-question, question-answering (QA) dataset with auditable gold answers. Every retrieval-augmented variant numerically improves strict vital-nugget recall over a naive BM25 baseline (63.8%), with the full corrective Agentic GraphRAG scoring (70.3%). The cross-encoder reranker contributes significantly to answer quality: removing it and allowing the LLM to score relevance drastically reduces strict vital recall by 32.8%. The graph channel and corrective loop contribute positively as expected, but the performance gains are marginal. Additionally, we also compare the performance of open-source and closed-source LLMs in final answer synthesis. We release the APS-Bench construction methodology, the six-layer evaluation harness, and the underlying codebase, along with the '/aps-rag' retrieval agent skill framework, to support reproduction and adoption at other facilities. Together, the deployed platform and its operations-grounded evaluation present a promising workflow for trustworthy, statistically grounded AI assistance in facility operations, transferable to other large scientific instruments.