Simular's Agent S4 evolves agent interfaces to boost OSWorld scores
Automated interface design shifts the bottleneck in computer-use agents from model quality to environment design, letting a small open model beat frontier systems on cost-adjusted performance. Builders of agent pipelines can apply the same trajectory-driven tool synthesis and population-based selection pattern to their own harnesses. The open-source release makes the method directly reproducible without proprietary APIs.
Simular's Agent S4 introduces a meta-agent that automatically designs interfaces for computer-use agents instead of relying on hand-built ones. The meta-agent reads the task agent's trajectories without success labels and writes new tools plus usage notes. With these evolved interfaces, an open 27B model scores 21.3% on OSWorld V2, 3.8 times higher than the same model with hand-built interfaces, exceeding GPT-5.5 at 13.0% and Claude Opus 4.8 at 20.6% at roughly 4.3% and 1.5% of their cost. Population-based selection keeps interfaces that generalise to unseen validation tasks, and on ScienceBoard the approach raises success from 41.4% to 58.6% with the same model. The code is open source.
Banger paper on evolving agent interfaces.
Pay close attention to this new trend if you build with agents.
Like optimizing harnesses, it's remarkable what you can achieve with agents that have access to better interfaces.
Computer-use agents usually act through interfaces that people design by hand, so an agent may type a spreadsheet cell by cell when one line of code would do. Simular's Agent S4 has a meta-agent that builds those interfaces instead. It reads the task agent's trajectories, with no success labels, and writes new tools and notes on when to use them.
With the evolved interfaces, an open 27B model reaches 21.3% on OSWorld V2, 3.8 times that of the same model with hand-built interfaces.
That is above GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%), at about 4.3% and 1.5% of their cost.
Population-based selection keeps the interface repositories that do best on validation tasks the meta-agent never sees, so the changes carry over to new tasks.
On ScienceBoard, the evolved interfaces raise success from 41.4% to 58.6% with the same model. The code is open source.
https://t.co/GugEMxwuzg
Chat with Paper: https://t.co/5stmPhni6v
- arXiv cs.AI10-20 17:47原文
