dair.ai 倡导企业自建模型评测体系：公共基准之外，生产环境需要 own evals
dair.ai 的 Elvis 在说：公共基准看看就行，模型能不能进生产得靠自己的 evals。他还把 evals 和 RSI 挂钩，说这是企业优化的地基，要自建不要外包。做模型选型的工程师和企业团队值得看看这条思路。
Elvis（dair.ai）在 X 上发文，主张企业不应依赖公共基准来做模型选型，而要在内部自建 evals。
他认为公共基准对研究和公开讨论仍有价值，但真正判断模型能否用于生产，只有靠针对自身业务的评测。文中还提出 evals 的作用不止于衡量可靠性，而是解锁新用例、以及支撑 RSI（递归自我改进）优化的关键基础设施，结论是"evals 要 own，不要 rent"。
dair.ai 表示已开始一项帮助企业内部搭建评测能力的业务，并提到与 Hamel Husain 等人的讨论。
Own your evals, folks! I have nothing against public benchmarks; they are still important for research and public discussions. However, the only way to truly trust which models to use in prod is to build your own evals. And it's not just about measuring reliability and model picking; evals are now crucial for unlocking new use cases and, eventually, RSI. Evals will be a key part of how any business will optimize with RSI. You don't want to rent this. You want to own this. Start building that expertise in-house. That is why, at @dair_ai , we have started an effort to help companies build these internally. Florian Brand @xeophon jammed with @HamelHusain a bit and since then, i cannot unsee of the state of evals as being similar to the military-industrial complex 🔗 View Quoted Tweet📊 14 ⚡
This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include 模型评测, Evals, dair.ai, RSI.
Source attribution: curated from elvis (@omarsar0).
