做 AI 评估或研究基准的团队会发现，DeepWeb-Bench 揭示了现有基准无法区分的模型能力差异——尤其是推导和校准的短板。建议关注其分能力族评估和来源溯源设计，这对理解模型真实研究能力很有帮助。
DeepWeb-Bench 是一个新的深度研究基准，旨在评估 AI 模型在开放网络上进行复杂研究的能力。与现有基准不同，该基准要求模型进行大规模证据收集、跨来源整合和长链条多步推理，难度显著提升。研究对九个前沿模型进行了评估，发现检索并非主要瓶颈（仅占12-14%错误），而推导和校准失败占70%以上。强模型和弱模型的失败模式不同：强模型主要因推导不完整出错，弱模型则因虚假精确性出错。该基准还揭示了模型在领域上的真实专长差异，跨模型一致性仅为0.61。
DeepWeb-Bench: A Deep Research Benchmark Demanding Massive Cross-Source Evidence and Long-Horizon Derivation
Deep research, in which an agent searches the open web, collects evidence, and derives an answer through extended reasoning, is a prominent use case for frontier language models. Frontier deep research products score high on existing benchmarks, making it difficult to distinguish their capabilities from current evaluation data alone. We introduce DeepWeb-Bench, a deep research benchmark that is substantially harder than existing benchmarks for the current frontier. Difficulty comes from three properties of the data itself: each task requires massive evidence collection, cross-source reconciliation, and long-horizon multi-step derivation. We represent these three sources of difficulty as four capability families (Retrieval, Derivation, Reasoning, and Calibration) and report results sliced by family. Every reference answer is accompanied by a source-provenance record with four disclosure levels and cross-source checks where available, making scores easier to audit against the underlying evidence. We evaluate DeepWeb-Bench on nine frontier models and report three findings: (1) retrieval is not the bottleneck, as retrieval failures account for only 12-14% of errors while derivation and calibration failures account for over 70%; (2) strong and weak models fail in qualitatively different ways, with strong models' errors dominated by incomplete derivation and weak models' by hallucinated precision; and (3) models exhibit genuine specialization across domains, with cross-model agreement of only rho = 0.61 and per-case disagreement reaching 18.8 percentage points. The public benchmark release includes the data, rubrics, and evaluation code.