Together AI Lifts Agent Task Completion From 67% to 96%
The notable detail is that most of the gain came from fixing documentation and API ergonomics, not from swapping models — a cheap lever many teams overlook. Engineers building agent products or running Claude Code in production can copy this three-config sandbox setup to isolate whether MCP servers or skills actually add value. It also shows Together AI doubling down on agentic workflows as a competitive differentiator in the developer tools market.
Together AI shared its internal methodology for evaluating coding agents, reporting that agent task completion improved from 67% to 96%. The workflow starts by defining roughly 10 representative tasks that mirror real usage of the product.
The team then runs agents — specifically Claude Code — inside sandboxes for each task, testing three configurations per task: a baseline setup, an MCP-enabled setup, and a setup using skills. After each run, engineers analyze the full agent trajectories to identify where tasks fail, then use those findings to improve documentation and API design rather than only tuning prompts.
The approach treats evals as a product feedback loop instead of a one-off benchmark score. For teams building agent-based developer tools, the takeaway is that failures often trace back to unclear docs or awkward APIs, and iterating on those surfaces can produce larger gains than changing the underlying model.
How we do agent evals at Together:
→ Define ~10 representative tasks → Run agents (Claude Code) in sandboxes → Test 3 configs (baseline, MCP, skills) for each task → Analyze trajectories and improve docs + APIs
Agent task completion improved from 67% to 96%! More below: https://t.co/51fUlbakvR
This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Together AI, Claude Code, agent evaluation, MCP and tools.
Source attribution: curated from Together AI (@togethercompute).