Hamel Husain 质疑公开基准测试质量，建议亲自跑分检验
Hamel Husain 提醒大家别盲信 benchmark，最好亲手跑一遍、看评分器错在哪，还分享了一个专门审查基准数据的网站。
工程师 Hamel Husain 在 X 上表示，行业对公开基准测试的依赖程度与其质量问题不成比例。他推荐了由 @xeophon 指引的一个独立第三方网站，该站专门审查基准测试并核查原始数据。他建议在信任任何基准前，先自己运行测试并仔细检查评分器判错的样本。他计划收集更多数据后写成文章。
I've been looking into public benchmarks and am astounded by the poor quality vs. how much we rely on them as an industry.
@xeophon pointed me to this excellent site https://t.co/xt5uUSyrcA which is an independent third party which scrutinizes benchmarks and "looks at the data" 💪
For example, here are some popular benchmarks we all rely on. Before trusting a benchmark its worth running them and inspecting examples the grader failed carefully, to get a sense of the quality.
I'm continuing to look more into this, and will write something up eventually once I've collected more data, but I think this is an important subject.
