DeepSeek V4.1 Flash benchmarked against coding agents
For developers, this matters because it shows DeepSeek V4.1 Flash's coding capabilities are robust and not tied to a specific agent framework, making it a versatile choice for different coding tasks.
The technical report for DeepSeek V4.1 Flash includes a section comparing its performance across different coding agent frameworks. It benchmarks the model against Claude Code, Codex, OpenCode, Pi, mini-SWE, and DSH on DeepSWE v1.1 and Terminal-Bench v2.1. Results show that the model performs well across various agents, with mini-SWE achieving the highest score of 74.2 on DeepSWE. The report also notes that DSH's internal modes have varying performance, and Claude Code's results are stable across different versions.
DeepSeek V4.1 Flash 的技术报告中，有一个实用的章节 5.3.4「Performance across agent scaffolds」 对比了 DeepSeek V4.1 Flash 在不同 Coding Agents 之间的 benchmark 差异，目的是为了说明模型不会依赖和耦合于某个特定的 Agent Harness Coding Agents 包括：Claude Code、Codex、OpenCode、Pi、mini-SWE 和 DSH；benchmark 包括 DeepSWE v1.1 和 Terminal-Bench v2.1 几个值得注意的评测数据： 1. 第三方框架并不弱。在 DeepSWE 上，为 Claude 模型设计的 Claude Code 拿到 69.8，高于 DSH PTC；极简的 mini-SWE 反而是全场最高的 74.2；Codex 和 OpenCode 最低。 2. DSH 内部三种模式差异不小：DeepSWE 上 Minimal 72.6 → Standard 70.5 → PTC 67.6，工具越多反而略降，工具更丰富的 harness 不一定更好，简单接口有时更稳。 3. Terminal-Bench 上各配置压缩在一个很窄的区间里（84.1–90.6），说明任务接近饱和时区分度下降。 4. Claude Code 测了 4 个版本，结果非常稳定（DeepSWE 68.4–69.8，均值 68.9；Terminal-Bench 均值 87.8），排除了 “碰巧挑了个有利版本” 的质疑。 huggingface.co/deepseek-ai/De… DeepSeek @deepseek_ai 🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6 🔗 View Quoted Tweet 💬 1 🔄 0 ❤️ 1 👀 405 📊 1 ⚡
- lmarena.ai09-10 15:32原文
