How-toSelected

Together AI 公开 Agent 评测方法:任务完成率从 67% 提升到 96%

Why we picked it

Together AI 把自家 Agent 评测流程全摊开了:10 个任务、3 种配置、沙箱跑 Claude Code,靠轨迹分析改文档,完成率从 67% 冲到 96%。自己搭 Agent 工作流的朋友可以直接抄这套作业。

Together AI 分享了内部跑 Agent 评测的完整流程:先定义约 10 个代表性任务,再把 Claude Code 放进沙箱环境里执行,每个任务测 3 种配置(baseline、MCP、skills),最后分析执行轨迹并据此改进文档和 API。这轮迭代让 Agent 任务完成率从 67% 提升到 96%,提升 29 个百分点。

Body

方法的关键在于用沙箱隔离运行环境、按配置对照测试,通过轨迹分析定位 Agent 失败的原因,再把改进落到文档和 API 设计上,形成可复用的评测闭环。

对开发者来说,这套流程提供了一个可以直接照做的 Agent 评测模板,尤其适合用 Claude Code、MCP 或 skills 搭建自动化工作流的团队参考。

Original · Together AI

Here's how we do agent evals at Together AI:

→ Define ~10 representative tasks → Run agents (Claude Code) in sandboxes → Test 3 configs (baseline, MCP, skills) for each task → Analyze trajectories and improve docs + APIs

Agent task completion improved from 67% to 96%!

Industry Coordinates & Source Verification

This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Together AI, Claude Code, MCP, agent evaluation.

Source attribution: curated from Together AI (@togethercompute).