Evals 进阶指南：Hamel Husain 与 Shreya Shankar 联手发布续篇
Evals have been coming up more and more in my conversations with podcast guests and PM friends. Nea...
做了 50+ 家 AI 公司评测的两位作者出了续篇，Ramp、Shopify、Cursor 怎么靠 evals 降本增效都写了，还送个插件。
Lenny Rachitsky 邀请 Hamel Husain 和 Shreya Shankar 撰写评测（evals）体系的进阶续作，两人此前服务过 50+ 家 AI 公司。文中引用多个实际收益案例：Ramp 把自动收据识别准确率从 35% 提到 83%，Shopify 的 AI 工作流构建器比原前沿模型方案快 2.2 倍且便宜 68%。Harvey 重构 AI 合同审查器后内部质量分接近翻倍，Cursor 调优 Auto Balance 路由，在成本降低 41% 的同时用户满意度更高。文章覆盖多数团队跳过的关键步骤、哪些环节该自动化，并附一个让编码 Agent 干重活免费插件。Lenny 还提到上周分享的 25 个 PM 职位中近半要求有写 evals 的经验。
Evals have been coming up more and more in my conversations with podcast guests and PM friends. Nea...
Evals have been coming up more and more in my conversations with podcast guests and PM friends. Nearly half of the 25 awesome PM job openings I shared last week ask for experience writing evals. And leading companies keep sharing what investing in evals bought them: — @tryramp took its automatic receipt collection from 35% to 83% accuracy. — @Shopify shipped an AI workflow builder that's 2.2x faster and 68% cheaper than the frontier-model setup it replaced. — @harvey__ai rebuilt its AI contract reviewer, nearly doubling its internal quality score. — @cursor_ai tuned its Auto Balance routing, with much higher user satisfaction at 41% lower cost. So I asked the 🐐s of evals, @HamelHusain and @sh_reya , to write an advanced sequel to their very popular "Building eval systems that improve your AI product." Drawing on their work with 50+ AI companies, they share the key step most teams skip, what you should (and shouldn't) automate, and a free plugin that lets a coding agent do most of the heavy lifting. Read it here: lennysnewsletter.com/p/advanced-eva… F 💬 23 🔄 12 ❤️ 148 👀 17456 📊 65 ⚡
