dair.ai 团队搞了个开源工具 ProofAgent-Harness，能给你的 Agent 上下文质量打分，提前知道它会不会幻觉、被攻击或乱用工具，比光看行为测试靠谱多了。
ProofAgent-Harness 是一个开源评估基础设施，通过多陪审员共识对 Agent 上下文进行七维评分，涵盖角色清晰度、护栏覆盖、指令一致性、工具模式质量、基础充分性、注入防护和令牌效率。实验表明，基础充分性分数预测幻觉抵抗能力，护栏覆盖分数预测操纵抵抗能力，工具模式质量分数预测工具使用准确性。该工具将上下文评分与行为指标和发布决策分离，独立作为 Agent 可靠性的领先指标。论文发布于 arxiv.org/abs/2607.14275。
// Agents Do Not Fail Alone // Very nice open-source eval tool to check agent reliability. Lots of...
// Agents Do Not Fail Alone // Very nice open-source eval tool to check agent reliability. Lots of cool ideas in there. (bookmark it) What's the problem? Context engineering has become central to building reliable agents but it remains almost entirely unmeasured. Instructions, tools, memory, retrieved knowledge, guardrails, and untrusted inputs all accumulate in the context, and when that gets weak the agent drifts, hallucinates, misuses tools, becomes vulnerable to injection, and burns tokens. This work validates context-engineering quality as an independent leading indicator of agent reliability. The measurement lives in ProofAgent-Harness, open-source infrastructure that scores context with multi-juror consensus across seven criteria, covering role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, and token efficiency. The context score is isolated from behavioral metrics and release decisions, so the prediction is not smuggled in. Holding frontier LLM agents fixed and varying only their operating context, each criterion predicts its matching outcome. Why you want to consider doing these checks: > Grounding sufficiency predicts hallucination resistance. > Guardrail coverage predicts manipulation resistance. > Tool-schema quality predicts tool use. Paper: arxiv.org/abs/2607.14275 Learn to build effective AI agents in our academy: academy.dair.ai 💬 9 🔄 6 ❤️ 20 👀 2069 📊 15 ⚡