SWE-OPD 论文:用在线蒸馏把大型教师模型的智能体编码能力迁移给小模型
SWE-OPD: On-Policy Distillation for Agentic Coding
用教师模型在学生访问的代码仓库轨迹上打分,Qwen3.5-9B 在 SWE-bench Pro 上白捡 3.5 个点,离线版还省 60% 以上 GPU 时。做小模型编码智能体、算力紧张的团队该看看,连跨 tokenizer 蒸馏和损失下降却输出重复的坑都讲清楚了。
arXiv 论文 SWE-OPD 研究如何用 on-policy distillation(OPD)把大型教师模型的智能体编码能力迁移到更小的学生模型。学生模型与可执行代码仓库交互,教师在学生访问的轨迹前缀上给出监督信号。实验覆盖 3 组教师-学生对、2 种训练数据来源和 3 个 SWE 基准。
具体结果方面,Qwen3.6-27B 的监督把 Qwen3.5-9B 在 SWE-bench Pro 上的解决率从 35.9% 提升到 39.4%。离线变体(用缓存的教师评分)不需要刷新轨迹,仍能超过学生基线,同时把端到端 GPU 时数削减 59.4%–72.6%,峰值 GPU 占用减半,只是精度随基准有所波动。跨 tokenizer 场景下,用 DeepSeek-V4-Flash-0731 蒸馏 GLM-4.5-Air,其 SWE-bench Pro 和 Multilingual 解决率分别从 35.8% 提到 39.7%、41.8% 提到 46.2%。
论文还发现训练损失下降不等于智能体表现提升:在某学生模型上,top-K reverse-KL 损失持续下降却出现严重重复输出,作者提出 support-mass 梯度项作为这一失败模式的可能机制解释。对需要在有限算力下部署编码智能体的团队,离线变体的成本数据和这一失败案例分析都有直接参考价值。
SWE-OPD: On-Policy Distillation for Agentic Coding
In this work, we study on-policy distillation (OPD) for transferring agentic coding capabilities from a large teacher model to a smaller student model. Prior work has focused primarily on reasoning and short-horizon agentic tasks. We present SWE-OPD, a minimal OPD study for software engineering, where the student interacts with executable repositories and the teacher supervises the prefixes visited by the student. Across three teacher--student pairs, two training sources, and three SWE benchmarks, we find that on-policy supervision transfers coding capability: supervision from Qwen3.6-27B increases the SWE-bench Pro resolve rate of Qwen3.5-9B from 35.9% to 39.4%. Refreshing trajectories is not strictly required either, as an offline variant with cached teacher scores still improves over the student baselines while cutting end-to-end GPU-hours by 59.4%--72.6% and halving peak GPU allocation, at benchmark-dependent accuracy. Furthermore, we extend the offline variant to corss-tokenizer scenarios: distilling GLM-4.5-Air from DeepSeek-V4-Flash-0731 raises its SWE-bench Pro and Multilingual resolve rate from 35.8% to 39.7% and from 41.8% to 46.2%, around 4% and 4.6% improvement, respectively. We also find that a decreasing loss, however, does not necessarily indicate improved agent performance: under top-$K$ reverse-KL loss, one student exhibits severe repetition despite a decreasing loss. We identify a support-mass gradient term as a potential mechanism underlying this failure.
This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include SWE-OPD, Qwen, GLM-4.5-Air, DeepSeek.
Primary source verification: published directly by arXiv: DeepSeek with first-party authority.