想知道 Arena 排行榜上的自动评分怎么算的吗？这篇解释了 AutoEval 如何用数百万人类投票训练奖励模型来给新模型打分，快且透明。
AutoEval 通过从实时 Arena 评估中采样的提示为新模型生成响应。它使用一个基于数百万对人类投票训练的点态奖励模型为每个响应打分，并将分数差转换为软投票。这些软投票通过相同的排名管道处理，结果直接显示在排行榜上作为初步 AutoEval 分数。闪电标记的行根据估计分数定位，但需获得足够实时人类投票验证后才能获得官方排名。
How does AutoEval produce Arena Score? For a new model to be evaluated, we generate responses on pr...
How does AutoEval produce Arena Score? For a new model to be evaluated, we generate responses on prompts sampled from live Arena evaluations. A pointwise reward model on millions of pairwise human votes to score each response and converts score differences into soft votes. We then pass those votes through the same ranking pipeline used for live human evaluations. The result appears directly on the leaderboard as a clearly marked, preliminary AutoEval score. The lightning-tagged row is positioned by its estimated score, but receives no official rank until enough live human votes arrive to validate it. 💬 1 🔄 2 ❤️ 10 👀 995 📊 2 ⚡