Papers

研究发现花哨的编程智能体框架更贵但效果没更好

Fancy harnesses make coding agents more expensive but not better, study finds

Why we picked it

有研究实测 Claude Code 和 OpenCode 这些编程框架,准确率差不多但价格能差 3 倍,选框架前值得看看

一项对照研究发现,在 Claude Code、OpenCode、mini-SWE-agent 等编程智能体框架之间切换,pass rate 的波动幅度与重复运行同一框架相差无几。成本差距却很显著:不同框架的单任务成本最多可相差 3 倍。成本上升的主因是每一步都要重发的 system prompt。

Original · Decoder

Fancy harnesses make coding agents more expensive but not better, study finds

A controlled study shows that switching between coding-agent harnesses like Claude Code, OpenCode, or mini-SWE-agent barely moves pass rates more than rerunning the same setup. Cost per task, however, can triple, driven mainly by the system prompt that gets resent with every step. The article Fancy harnesses make coding agents more expensive but not better, study finds appeared first on The Decoder .