RubricAtlas：从专家研报挖掘类型级评估标准，评深度研究智能体
RubricAtlas: Mining Type-Level Rubrics from Expert Reports to Evaluate Deep Research Agents
把 3000 多份金融研报提炼成评估深度研究智能体的打分标准，还能查到每条评分依据，做评估的人可以看看。
RubricAtlas 从 3,079 份中文金融研报中提炼出 99 种研究类型共用的分析性要求，用于评估深度研究智能体产出的报告。该框架把二元检查点关联到报告段落，并将分析完整性与论证有效性分开打分。在十款智能体上，两条评审管线的检查点一致性达 κ = 0.71，排名一致性达 Kendall τ = 0.87。实验显示单份研报仅覆盖其类型约一半的分析问题，从不同研报选取要求会改变评分与排名。删除受控内容会拉低对应分析项得分，而注水内容不带来分数上升。
RubricAtlas: Mining Type-Level Rubrics from Expert Reports to Evaluate Deep Research Agents
Deep research agents conduct multi-step Web searches and synthesise retrieved information into analytical reports. Evaluating these outputs requires checking which analyses they deliver, beyond the presence of relevant facts. Query-specific binary rubrics make individual requirements explicit, but task-level checklists may leave the overall analytical structure underexamined. We introduce RubricAtlas, which derives type-level analytical requirements from expert reports on different objects within the same research type and organises detailed checks around recurring analytical questions. The requirements give related tasks a shared standard for what to analyse, without prescribing their answers. Our framework links binary checkpoints to report passages and evaluates completeness separately from argument validity. Built from 3,079 Chinese financial research reports, RubricAtlas covers 99 research types. Reference-report selection experiments show that individual reports cover only about half of their type's analytical questions, and that selecting requirements from different reports changes evaluation scores and agent rankings. Across ten agents, two judge pipelines achieve checkpoint agreement of $κ$ = 0.71 and agent-ranking agreement of Kendall $τ$ = 0.87. Controlled deletions reduce scores on the affected analyses, while padding produces no detectable increase. These findings support type-level rubrics for comparing the analytical completeness of Web deep research outputs, with individual decisions traceable to report evidence.