How-toMulti-source

Vercel:你的下一批用户是 agent,evals 是面向它们做产品的方式

Why we picked it

Rauch 在 Vercel 博客里讲他们怎么做 agent evals:看 agent 怎么用你的产品、上线前测改动、新模型发布时做对比。做 API 或开发者产品的人值得照着搭一套自己的评估流程。

Guillermo Rauch 提出,传统的用户调研、用户旅程图正被新的开发范式取代:产品的主要使用者从人类变成了 agent,开发者要为它们做产品和优化体验。Vercel 发布的 agent evals 方法包含三个核心动作:观察 agent 如何实际调用你的产品、在上线前测试改动带来的影响、在新模型发布时对比不同模型的表现。

Body

这种做法的机制是把 evals 变成软件工程循环的一部分,替代人工访谈得到的设计灵感——agent 调用接口的行为是可测量、可回归测试的,改动是否破坏了 agent 的使用流程可以通过评估集验证。Vercel 在博客中给出了具体的 evals 实践细节。

对开发者而言,这一转变意味着接口稳定性、文档质量和错误信息的设计目标都要面向机器用户重新校准;产品方需要像维护 CI 一样维护 evals 基线,以适配持续更新的各家模型。

Original · Guillermo Rauch

The old way: users are human, you run user interviews, get inspired, mock up user journeys. The new way: users are agents, you run evals, evals inform the software engineering loop. Good read on how we do evals at Vercel. Vercel @vercel Your next users are agents. Evals are how you build for them. ▪︎ See how agents use your product ▪︎ Test changes before you ship ▪︎ Compare models as they launch vercel.com/blog/agent-eva… 🔗 View Quoted Tweet📊 33 ⚡

Industry Coordinates & Source Verification

This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Vercel, agent, evals, MCP and tools.

Source attribution: curated from Guillermo Rauch (@rauchg). Cross-source corroboration: 3 independent sources have covered this development (including Vercel AI, Lenny Rachitsky), confirming industry-wide signal strength.