做Agent生产的人快收藏:它把41种故障按交互边分类,修bug能直接判断是模型问题还是脚手架问题。
该论文提出一种 Agent 故障归因框架,将 41 种失败模式映射到模型、工具、记忆、环境等组件之间的交互边上。这41种模式均标出修复责任方,多数故障发生在模型与脚手架(harness)的接缝处。在四个前沿模型上,最强评判者对人类分类标签的 Cohen's kappa 达 0.76,可自动化标注生产轨迹。论文编号 arxiv.org/abs/2607.28802,适合生产环境构建智能体的团队参考。
// Model or Harness // Great paper if you are building with agents in production. (bookmark it) I...
// Model or Harness // Great paper if you are building with agents in production. (bookmark it) It organizes 41 agent failure modes by the interaction they originate in. Each mode gets assigned to an edge between two components (model, harness, user, tools, memory, environment) plus a fault side naming where the repair belongs. Attributing failures to edges rather than to single components matches how agent bugs actually present. Most of them live in the seam between a model and its scaffolding. The schema also holds up under automation. Across four frontier models, the strongest judge reaches Cohen's kappa of 0.76 against human category labels, so the labeling can run continuously over production traces instead of one postmortem at a time. Harness engineering became the main lever for agent builders this year without a shared vocabulary for where a harness bug ends and a model bug begins. Paper: arxiv.org/abs/2607.28802 Track more trending AI papers in our academy: academy.dair.ai 💬 5 🔄 8 ❤️ 62 👀 4868 📊 28 ⚡