法律AI的幻觉问题一直难量化，这个基准把检索和生成拆开评估，做法律NLP或合规系统的团队可以直接用来测试自己的RAG管线。
法律领域对检索增强生成（RAG）系统的可靠性要求极高，但现有基准缺乏细粒度评估，且多为英文、面向专家。研究者提出ClaimRAG-LAW数据集，支持法语和英语，覆盖专家与非专家用户，包含多样问题类型。通过细粒度评估框架分析现有法律RAG系统，揭示了检索、生成及声明级分析的局限性。该工作为法律AI的可靠性评估提供了更精准的工具。
Fine-grained Claim-level RAG Benchmark for Law
The rapid progress of large language models (LLMs) is shifting semantic search toward a question-answering paradigm, where users ask questions and LLMs generate responses. In high-stake domains such as law, retrieval-augmented generation (RAG) is commonly used to mitigate hallucinations in generated responses. Nonetheless, prior work shows that RAG systems, whether general-purpose or legal-specific, still hallucinate at varying rates, making fine-grained evaluation essential. Despite the need, existing evaluation frameworks for legal RAG systems lack the granularity required to provide detailed analysis of retrieval and generation performance separately. Moreover, current benchmarks are largely English-only and centered on legal expert queries, overlooking non-expert needs. We introduce ClaimRAG-LAW, a comprehensive dataset for legal RAG that supports French and English, targets both experts and non-experts, and includes diverse question types reflecting realistic scenarios. We further apply a fine-grained evaluation framework of state-of-the-art legal RAG systems, revealing limitations in retrieval, generation, and claim-level analysis in the legal domain.