Anthropic 的产品主管亲自聊他们怎么做模型评估，说 Evals 就像新版产品需求文档，对做 AI 产品和模型的朋友很有启发。
Anthropic 研究产品主管 Dianne Penn 在访谈中提出“Evals 是新的 PRD”这一观点。她于 2023 年加入 Anthropic，作为首位技术 PM，参与了从 Claude 2 到 Mythos 的每个 Claude 模型发布。访谈涵盖 AI 研究角色如何运作、前沿模型安全措施（如 35:18 处）、以及如何构建有效的评估体系。Lenny Rachitsky 主持讨论，还涉及 AI 时代招聘、产品经理角色演变等话题。
Anthropic's Head of Product for AI research: Evals are the new PRDs
Anthropic's Head of Product for AI research: Evals are the new PRDs Your browser does not support the video tag. 🔗 View on Twitter Lenny Rachitsky @lennysan Dianne Penn is Head of Product for Research and Labs at @AnthropicAI . She joined in 2023 as their first technical PM (when there were just five product engineers) and has helped ship every Claude model from Claude 2 to Mythos. We discuss: 02:31 Early Anthropic 08:55 The two biggest inflections 13:50 Inside the exponential 20:02 Token maxing 23:30 Anthropic Labs and the incubation model 27:30 How the AI research role works 31:35 How to become a top AI researcher 35:18 Frontier model safeguards 39:38 Hiring in the AI era 44:16 Building evals 47:48 Evals are the new PRDs 49:55 The importance of hands-on leadership 52:46 Finding joy in AI 58:10 How Dianne uses Claude 01:01:05 Avoiding overreliance on AI 01:03:50 The constitution that makes Claude better 01:07:11 AI writing and verification 01:11:40 Where human brains will continue to be valuable 01:14:10 Navigating AI with kids 01:16:26 Alignment, the future of the PM role, and burnout 01:21:54 Lightning round and final thoughts Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet 💬 3 🔄 1 ❤️ 9 👀 1959 📊 4 ⚡