做 AI 代理评估或可靠性研究的团队会感兴趣——DRIFT 框架直接解决了「只看结果不看过程」的盲区,建议点开看看方法细节。
论文研究深度研究代理在长轨迹中的错误定位问题,指出仅靠最终答案评估无法揭示轨迹中的不可靠部分。作者收集了 2,790 条真实轨迹,通过 LLM 辅助专家标注构建了 TELBench 基准,包含 1,000 个实例用于识别错误跨度。提出 DRIFT 框架,以声明为中心审计代理轨迹,追踪声明并检查证据支持,在多个模型和审计框架上将跨度级错误定位和首次错误准确率提升最多 30 个百分点。该工作为深度研究代理的可靠性提供了过程级视角。
Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories
Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis. Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable. We study span-level error localization for deep-research agents. We collect 2,790 real trajectories from two agent frameworks, three backbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmful error spans through LLM-assisted expert review. From these annotations, we build TELBench, a 1,000-instance benchmark for identifying error spans among normal exploration, failed searches, tentative hypotheses, and harmless noise. We further propose DRIFT, a claim-centric auditing framework that tracks agent claims, checks their support in trajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path. Experiments across model families and auditing frameworks show that DRIFT improves span-level error localization and first-error accuracy by up to 30 percentage points. Our work provides a process-level view of reliability in deep-research agents.