Hamel Husain 批评通用基准文化，主张做产品专属评测
Hamel Husain 是评测领域的知名实践者，这次他把矛头对准'给社区做基准'的说法，认为团队更该写产品专属评测。做 AI 产品、纠结选型或排名的工程师和创业者看完会很有共鸣。
开发者 Hamel Husain 在 X 上发文，批评当前围绕公开基准的争论，认为部分回应暴露了谁在追求事实、谁在维护既有利益。
他指出一个流行说法——'别抱怨，加入社区把基准做好'——并反驳称开源契约在此不成立，没人会免费钻进数据堆里为他人盈利。他的建议是把精力转向直接测试自家产品的产品专属评测（product-specific evals），而非追逐通用基准排名。
他同时澄清 Artificial Analysis（AA）对此态度开放，批评并非针对 AA。
Responses this revealed who is truth seeking vs who is desperately trying to hold on to easy money
The other psyop is “instead of complaining, join the community to make these benchmarks better” but I’m sorry that open source compact doesn’t hold here, people not gonna go into the data mines for free so someone else can profit. You are much better off focusing on product specific evals that directly test your products.
By the way AA seems really open minded about this, so this is not about them!
This report is categorized under AITOP's "Industry" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Hamel Husain, Artificial Analysis, 基准评测, evals.
Source attribution: curated from Hamel Husain (@HamelHusain).
