The design lesson is that multi-objective optimization in agent pipelines can backfire when applied too early — deferring the cost criterion until final selection preserves valuable intermediate candidates. Builders running agent harness or prompt searches should add repeated runs and a cost axis to their selection step. The arXiv paper (2611.02079) and dair.ai's Chat with Paper make it easy to replicate the setup.
#GAIA2
Choosing training tasks by remaining failures, rather than a fixed list, cuts optimiser spend by roughly 78% while lifting pass rates — task selection is a bigger lever than prompt or tool patching. Builders who auto-tune agent harnesses can redirect their run budget toward unresolved failure patterns. GAIA2 and Terminal-Bench 2.0 numbers give a direct benchmark for comparing harness optimisation setups.
The key insight is that optimizer quality alone is not the bottleneck in agent harness training — scenario scheduling is, and treating failure patterns as a bandit problem recovers gains the fixed-order baseline leaves on the table. Builders training coding or task agents can apply the same scheduling idea without changing their optimizer. Compared with GAIA2's 4.4-point gain, the 7.5-point jump on Terminal-Bench 2.0 suggests the method helps most where failure modes are more diverse.