管理咨询团队和依赖AI做深度分析的开发者会震惊——三个最先进的DRA在专家级任务中通过率不到22%，且各有致命短板。想避免被AI的自信输出误导，建议仔细看这篇评测的失败模式分析。
研究人员发布了针对深度研究代理（DRA）在管理咨询场景下的评测基准，包含42个专家撰写的任务，每个任务有平均13.8个确定性验证器和五维度0-3分专家评分。评测了Claude Opus 4.6、OpenAI o3-deep-research和Google Gemini 3.1 Pro，三者通过联合阈值（专家评分≥2.5且验证器通过率≥80%）的接受率均很低：Gemini 21.4%，o3和Claude仅9.5%。各模型失败模式不同：Claude输出最可靠但虚构最多，o3推理最清晰但遗漏章节和传播算术错误，Gemini表现两极分化。该基准通过嵌入认知陷阱来惩罚表面模式匹配，揭示了当前前沿DRA在专业分析任务上的严重不足。
Evaluating Deep Research Agents on Expert Consulting Work: A Benchmark with Verifiers, Rubrics, and Cognitive Traps
Frontier deep research agents (DRAs) plan a research task, synthesize across documents, and return a structured deliverable on demand. They are being deployed in enterprise workflows faster than they are being evaluated. Existing benchmarks measure factual recall, single-hop QA, or generic agentic skill, missing the multi-document, decision-grade work DRAs are deployed to produce. We introduce a benchmark targeting the structured analytical deliverables that fill a management consultant's typical week. We grade three frontier agents, namely Claude Opus 4.6 with web search, OpenAI o3-deep-research, and Google Gemini 3.1 Pro deep-research, on 42 SME-authored prompts. Each of the 126 responses is scored on two layers: deterministic ground-truth verifiers (mean 13.8 per task) and a five-criterion 0-3 SME rubric, composed into a Verifier-Rubric Score (VRS) on 0-100. Most prompts embed cognitive traps that penalize surface-pattern matching. Acceptance under our joint threshold (rubric mean >= 2.5 and verifier rate >= 80%) is uniformly low: Gemini 21.4%, o3 9.5%, Claude 9.5%. Mean VRS scores agree with published rubric-based benchmarks (our top 62.6 vs. APEX-v1 64.2, ProfBench 65.9, ResearchRubrics < 68%), validating the rubric construct. ACCEPT rates sit below APEX-Agents' MC-segment Pass@1 band (12.3-22.7%) on dedicated DR agents; our floor is three points lower despite the harness advantage, opened by stricter conjunctive grading and trap design. Each agent fails distinctively. Claude produces the deliverable most reliably (4.5x the others' rate on file-required tasks) but carries the highest fabrication signature. o3 has the cleanest reasoning average yet drops required sections and propagates arithmetic errors. Gemini is bimodal, with the highest acceptance rate alongside the most zero-scored rubric cells.