做 AI 模型评估或选型的团队，DeepSWE 能帮你看到模型在长周期软件工程任务上的真实差距，建议关注这个新基准。
Datacurve 推出 DeepSWE，一个更严格的编程基准测试，旨在揭示领先模型之间的真实差距。GPT-5.5 得分 70%，而 GPT-5.4 为 56%，Claude Opus 4.7 为 54%，差距在旧基准中常被掩盖。DeepSWE 使用原创任务而非公开 GitHub 问题，避免模型训练时见过答案。其提示词长度仅为 SWE-bench Pro 的一半，但解决方案需要 5.5 倍代码量和约 2 倍输出 token。评分方式也不同，DeepSWE 检查请求行为是否真正实现，而非仅依赖合并 PR 的测试。
Datacurve launches DeepSWE, a tougher coding bench…
Datacurve launches DeepSWE, a tougher coding benchmark made to show where leading models truly separate.
GPT-5.5 hits 70%, while GPT-5.4 reaches 56% and Claude Opus 4.7 reaches 54%, making a gap that older benchmarks largely hid.
Its a long-horizon software engineering benchmark.
- DeepSWE differs from older coding benchmarks in the source of the exam: older tests often reuse public GitHub issues and PRs, while DeepSWE uses original tasks, so models are less likely to have seen the answer during training.
- The work is also bigger even when the prompt is shorter, because older tests often tell the model what area to touch, while DeepSWE makes the agent search the repo, understand the design, edit multiple files, and avoid breaking old behavior.
On DeepSWE, prompts are half the length of SWE-bench Pro's, yet solutions require 5.5x more code and ~2x more output tokens.
- The grading is different too, because many older benchmarks reuse tests from one merged PR, while DeepSWE checks whether the requested behavior actually works, even if the model solves it in a different valid way.
- Philipp Schmid05-27 09:17原文