跑Agent生产评估的看过来，这篇把人类专家放在上游，评估资产能复用，不用每次输出都人工审，效率高多了。
论文提出Human-on-the-Bridge方法，将人类判断前置到可复用的评估资产中，用于生产环境下的AI Agent评估。Agent作为行为系统需要跨轮推理、调用工具、保持上下文和遵循策略，现有方法如静态Benchmarks、LLM-as-judge、红队测试各有局限。该方法由专家在测试前策划可复用的评估智能，而非在循环中逐条审查输出。论文编号2606.16871，展示了提升可扩展性的具体路径。
>> Scalable Evaluation for AI Agents << If you run agent evaluation in production, this one is wort...
>> Scalable Evaluation for AI Agents << If you run agent evaluation in production, this one is worth your time. It shows that front-loading human judgment into reusable evaluation assets is useful. But why? Agents reason across turns, call tools, hold context, follow policies, and act under uncertainty, so they have to be judged as behavioral systems. Current methods each give a fragment. Benchmarks measure fixed capabilities, human review preserves judgment but does not scale, LLM-as-judge inherits the evaluator design problem, red teaming is episodic, and trace audits need explicit evidence rules. Human-on-the-Bridge puts human expertise upstream, where experts curate reusable evaluation intelligence before testing rather than reviewing each output in the loop. Paper: arxiv.org/abs/2606.16871 Learn to build effective AI agents in our academy: academy.dair.ai 💬 7 🔄 5 ❤️ 18 👀 1297 📊 11 ⚡