Choosing training tasks by remaining failures, rather than a fixed list, cuts optimiser spend by roughly 78% while lifting pass rates — task selection is a bigger lever than prompt or tool patching. Builders who auto-tune agent harnesses can redirect their run budget toward unresolved failure patterns. GAIA2 and Terminal-Bench 2.0 numbers give a direct benchmark for comparing harness optimisation setups.
#ActiveSaddler
2 stories · 0 in 7 days · 0 in 30 days
Stories, products and papers tagged "ActiveSaddler", newest first.
Oct 5
ActiveSaddler: Microsoft Paper on Automated Agent Harness Optimisation
Oct 4
ActiveSaddler boosts agent harness optimizers via adaptive scenarios
The key insight is that optimizer quality alone is not the bottleneck in agent harness training — scenario scheduling is, and treating failure patterns as a bandit problem recovers gains the fixed-order baseline leaves on the table. Builders training coding or task agents can apply the same scheduling idea without changing their optimizer. Compared with GAIA2's 4.4-point gain, the 7.5-point jump on Terminal-Bench 2.0 suggests the method helps most where failure modes are more diverse.