Arena.ai 搞了个新评估 AutoEval，用千万用户偏好打分，几小时出结果，比人工快多了，排行榜直接看分数。
Arena.ai 发布新评估方法 AutoEval，使用基于数百万真实用户偏好训练的奖励模型对 AI 模型进行排名。该方法与人工评估高度对齐，速度从数天缩短至数小时。AutoEval 支持 Text、Vision、Image 和 Code 等多个基准。新模型将直接在排行榜上显示 AutoEval 预估分数，加速社区获取结果。
Read more about AutoEval scores and our methodology: https://t.co/2kEg4iZUSG
Read more about AutoEval scores and our methodology: x.com/arena/status/2… Arena.ai @arena Today we’re launching AutoEval: a new evaluation methodology that ranks models using reward models based on millions of real Arena user preferences. Highlights: - High-quality evaluation signals calibrated on real preference data - Strong alignment with live human evaluations - Evaluations that are several orders of magnitude faster (hours instead of days) - Support for Text, Vision, Image, and Code Arena AutoEval enables us to evaluate newly launched models much faster and share results with the community sooner. We’ll now show AutoEval estimated scores for new models directly on the leaderboard. More details in the thread. 🧵 🔗 View Quoted Tweet 💬 0 🔄 0 ❤️ 2 👀 1594 ⚡
- 向阳乔木17:43原文