研究:编程智能体普遍存在"提前解对、迟迟不停"的终止滞后问题
Termination Lag: Coding Agents Solve Early but Stop Late
做 agent 或跑 SWE-bench 的人该看看这篇。它发现不少模型第一次改代码就对了,之后一半步数在白跑,甚至有 1–8% 的运行把对的补丁改坏。想省 token、提高评测准确度,这个指标很实用。
arXiv 论文《Termination Lag: Coding Agents Solve Early but Stop Late》量化了编程智能体从首次产出正确补丁到结束运行之间的多余步数。研究覆盖五个模型、两个 agent 框架和三个 SWE-bench 数据集,逐条记录并测试运行中生成的每个补丁。
数据显示,38–82% 的成功运行在第一次源码编辑时就已经得到正确解,但智能体仍在之后继续消耗大量算力:在 SWE-bench Verified 和 Multilingual 上,Qwen3-Coder 成功运行中解出后的步骤占中位数 35–37%,Laguna-XS.2 占 54–60%。这些多余步骤大多花在验证和继续探索代码仓库上。
更严重的是,1–8% 的案例里,后续活动导致智能体改掉了先前正确的补丁,最终提交反而错误。这意味着评测只用最终提交打分会低估真实能力,也给 agent 运行成本控制指出一个可优化点:提前判定首次正确解并终止运行。
Termination Lag: Coding Agents Solve Early but Stop Late
Coding agents can solve a task well before they stop working on it. We characterize \emph{termination lag}--the number of steps between an agent's first correct patch and the end of its run--across five models, two agent frameworks, and three SWE-bench datasets by recording and testing every distinct repository patch generated during execution. Our analysis reveals that 38--82% of successful runs achieve a correct solution on their initial source-code edit. However, agents continue to expend substantial effort after correct patches, with post-solution steps accounting for a median of 35--37% for Qwen3-Coder and 54--60% for Laguna-XS.2 in successful runs on SWE-bench Verified and Multilingual. Agents spend most of these additional steps verifying their work and exploring the repository. In a small fraction of cases (1--8%), this continued activity leads agents to undo their earlier correct patch, leaving the final solution incorrect.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include SWE-bench, Qwen3-Coder, 编程助手, 智能体.
Source attribution: curated from arXiv cs.AI (@Musa Cim, Burak Topcu, Mahmut Taylan Kandemir).