Notion推出Knowledge Board：基于真实任务的模型评测
Notion搞了个新评测，用真实用户任务测模型，看谁成功率高又省时省钱，比传统benchmark更实用。
Notion发布了Knowledge Board评测项目。它从实时流量中随机抽取少量匿名数据，并在各模型间均匀分配。该评测衡量真实任务的成功率、所需时间和成本，而非依赖固定基准。其目的是帮助用户根据自身工作负载找到性价比合适的模型。
Most model evaluations rely on fixed, curated tasks. They’re useful, but they don’t show how models perform on the messy, varied work people actually give them. That’s the motivation behind @NotionHQ ’s Knowledge Board. It uses a small, random, anonymized sample of live traffic, split evenly across models, to measure whether real world tasks are successfully resolved, as well as the time and cost required. Rather than offering a typical leaderboard, the project aims to answer a more practical question: Which model offers the right tradeoff for the work you’re trying to do? labs.notion.com/knowledge-board 💬 0 🔄 2 ❤️ 11 👀 1640 📊 3 ⚡
