PapersSelected

Northwestern 与 Google 论文提出 Harness2:无答案密钥下让智能体自我改进 harness

Why we picked it

做智能体的朋友值得看:harness 调优通常是手工试错,这篇论文让编辑模型自己提修改、实测验证再入库。GLM-5 一次迭代就涨 11.5 分超过 Gemini 3.5 Flash 的原生 harness,跳过实测反而掉分,这个对照很说明问题。

Northwestern 与 Google 的新论文提出 Harness2,让智能体 harness(包裹模型的提示词、工具、技能与记忆)在新任务上自我改进,无需答案密钥。其机制是先运行智能体,由编辑模型提出 harness 修改建议,再重跑任务验证哪些修改真正改善了行为,每个任务的经验存入库中,用于指导下一次编辑。

Body

实验结果显示,单次改进就让 GLM-5 在 WorkBuddy-Bench 上提高 11.5 分,超过使用原生 harness 的 Gemini 3.5 Flash。对照组中,跳过真实测试直接编辑 harness 反而使编程得分从 73.4 降到 63.2,说明用实测筛选修改是关键环节。

论文的实用结论是:把 harness 当作需要按任务迭代修订的对象,只保留在真实运行中改变了智能体行为的修改。对手工调优 harness 的团队来说,这种自动化验证流程可减少人工试错成本。

Original · rohanpaul_ai

New Northwestern and Google paper shows that agent harnesses can improve themselves on new tasks without answer keys, by test-running edits and keeping what helps.

An agent's harness is the prompts, tools, skills, and memory wrapped around the model. Tuning it by hand is slow, and new real-world tasks come with no answer key to tune against.

Harness2 runs the agent, has an editor model propose harness fixes, then reruns the task to see which fixes improved the agent's behavior. Lessons from each task go into a library that guides edits on the next one.

A single improvement step raised GLM-5's WorkBuddy-Bench score by 11.5 points, past Gemini 3.5 Flash on its stock harness. Editing the harness without those test runs instead dropped coding scores from 73.4 to 63.2.

Treat your agent's harness as something to revise per task, and keep only the edits that change its behavior on real runs.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Harness2, GLM-5, agents, WorkBuddy-Bench.

Source attribution: curated from rohanpaul_ai (@rohanpaul_ai).