Cursor releases private benchmark CursorBench 4.0
For developers, this matters because it provides a more realistic and difficult benchmark to test AI coding assistants, moving beyond public benchmarks that can be gamed.
SWE-bench and other public benchmarks have been 'gamed', so Cursor has updated its private benchmark to version 4.0. Claude Fable 5.1 and Opus 5 are expensive but perform best. Grok 4.6's performance is not outstanding, showing little advantage over Muse Spark 1.3 and being more expensive. The new benchmark uses a method called 'reverse question generation' from real conversations, called Cursor Blame, to trace code back to its original request. It includes more complex tasks like monorepo workspaces and production logs, and intentionally retains ambiguity in task descriptions. The benchmark evaluates correctness, code quality, efficiency, and interaction behavior, with a separate online test to catch issues that offline tests miss.
SWE-bench 等公开基准被“刷爆”，Cursor 更新发布私有基准 CursorBench 4.0 · Claude Fable 5.1、Opus 5 贵但确实最强 · GPT-6 Astra 不参与评测（原因你懂） · Grok 4.6 表现不突出，和 Muse Spark 1.3 相比优势不大，且更贵 · Lee 表示 Grok 4.7 会有很大提升 cursor.com/cn/cursorbench 再看看 CursorBench 方法论：从真实会话中“反向出题” 核心机制是 Cursor Blame：将已提交的代码追溯回生成它的智能体请求，天然形成“开发者查询—标准答案”配对。配套设计包括： · 防泄漏：大量任务来自内部代码库和受控来源，且每隔数月整体更换题库； · 任务更重：从初版到 3.0，任务规模（代码行数、涉及文件数）约翻倍，明显大于 SWE-bench 各变体，并引入 monorepo 多工作区、排查生产日志、长时实验等难题； · 刻意保留歧义：任务描述保持简短、信息不充分，模拟开发者真实的表达方式，而非为方便评分而人为消除不确定性。 评测理念：线上-线下双闭环 CursorBench（线下）衡量正确性、代码质量、效率、交互行为多个维度；线上评估则在真实流量上做受控实验，专门捕捉“离线得分高、但开发者实际体验更差”的回归，这类问题是纯离线评测的盲区。 Cursor 给了一个消融实验案例：完全移除语义搜索工具后，才准确定位其价值集中在大型代码库的仓库级问答场景。 Lee Robinson @leerob We just rolled out CursorBench 4.0! It includes new tasks for how well models follow instructions, work on challenging projects over time, and is more difficult than before (so all models score lower). 🔗 View Quoted Tweet 💬 0 🔄 1 ❤️ 1 👀 532 📊 1 ⚡
