Chinese contentNoam Brown 赞赏一份基准测试方法，并邀请 Dwarkesh 参与 OpenAI 评测设计
OpenAI 的 Noam Brown 公开夸一份基准的方法论，还顺手挖 Dwarkesh Patel 去做评测设计，想看哪种评测能入他的眼可以点进链接。
#评测设计
2 stories · 1 in 7 days · 1 in 30 days
Stories, products and papers tagged "评测设计", newest first.
Nov 7
Sep 29
Anthropic open-sources build-eval and hillclimb workflows for Claude Code
The practical value lies in the grader discipline: rubrics written as verifiable statements instead of 1–5 scales, pairwise comparison with randomised order, and programmatic checks whenever output is constrained. Engineers building evals for coding agents can adopt these as review checklists and reuse the open-source Skills directly in Claude Code. The four reliability criteria, borrowed from psychometrics, give teams a concrete diagnostic for deciding when an eval has lost discrimination and optimisation should shift to cost.
