Google paper: gated modular edits beat full agent harness rewrites
The cost gap, roughly 10x tokens for a 2.5-point accuracy gain, means gated modular editing is the only economically viable route for most teams running self-improving agents. Builders of agent frameworks and coding harnesses should adopt validation-gated edits and code-enforced caps rather than prompt-based rules, which the paper shows can collapse accuracy. Teams using Gemini models or SWE-bench-style evaluation pipelines get a concrete recipe for tuning harnesses without runaway inference bills.
A new Google paper examines self-improving agent harnesses, the code around a model that manages context, runs tools, and verifies work before stopping. It finds that letting an optimizer freely rewrite the harness matches accuracy on paper but costs nearly 10 times more tokens per task than a constrained, modular approach.
The researchers let an optimizer model improve a 142-line coding harness by editing one of its four parts at a time. Each edit had to pass a validation set and a token cap or it was rejected. On 40 held-out SWE-bench Lite tasks with Gemini 3.8 Flash, the gated harness solved 40.0% versus 5.0% for the seed harness, at 46.7k tokens per task. A full unchecked rewrite solved slightly more (42.5%) but consumed 451.6k tokens per task, while enforcing rules through prompts alone dropped accuracy to 2.5%.
The practical takeaway is that harnesses tuned for one model often stall or waste tokens on another, so teams building self-improving agents should gate every change on held-out tasks and token cost, and enforce constraints in code rather than prompts.
New Google paper finds that an unchecked full rewrite of an agent harness matched accuracy but cost nearly 10 times more tokens per task than gated modular edits.
that self-improving agent harnesses work best when they change 1 code module at a time and must pass validation.
The harness is the code around a model that manages context, runs tools, and checks work before stopping. A harness tuned for 1 model often stalls or wastes tokens on another.
They let an optimizer model improve a 142-line coding harness by editing 1 of its 4 parts at a time. Each edit had to pass a validation set and a token cap, or it was rejected.
On 40 held-out SWE-bench Lite tasks with Gemini 3.8 Flash, the gated harness solved 40.0% against 5.0% for the seed, using 46.7k tokens per task. A full unchecked rewrite solved 42.5% but used 451.6k tokens, and prompt rules dropped accuracy to 2.5%.
If you let agents tune their harness, gate each change on held-out tasks and token cost, and enforce rules in code, not prompts.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Google, SWE-bench, agent harness, agents.
Source attribution: curated from rohanpaul_ai (@rohanpaul_ai).