为什么评测才是 AI 公司真正的护城河
Saravia 说 post-training 会越来越容易，公司都得像实验室一样自建智能栈，但真正的护城河是评测。做 AI 产品的开发者值得看看他对 evals 的判断。
Elvis Saravia（omarsar0）发推主张 AI 公司应该自建并拥有自己的评测体系。他提出越来越多公司将被迫像 neolab 一样运作，即自建 intelligence stack，随着 post-training 门槛降低，这一趋势会加速。他强调可以正确测量的东西才能更容易被优化，没有评测工厂就没有软件工厂。
他的核心论点是：在任何人都能构建产品的时代，taste（品味）比以往更重要，而评测正是把 taste 固化下来的方式。作者提到自己在这个领域已工作超过 10 年，过去一年也在围绕这一方向进行开发。
Build and own your evals, folks. Why do I keep persisting in this? I do think more and more companies will be forced to operate like a neolab. In other words, more companies will need to own and build their own intelligence stack. This is already happening across many domains. As post-training becomes more accessible, I expect the numbers to accelerate. Access to compute will be a real challenge, but the industry always figures things out. But I think the real moat is in developing evals. I've been working in this space for over a decade, and one thing never changes: what can be properly measured can be more easily optimized. That is crucial in a world of RSI. No software factory without an eval factory. In a world where anyone can build anything, taste matters more. That is codified in evals and further codified in the intelligence stack. I have been building around this in the past year. If you want to chat about this, let's connect.📊 7 ⚡
This report is categorized under AITOP's "Industry" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include 评测, post-training, intelligence stack.
Source attribution: curated from elvis (@omarsar0).
