Browser Use benchmark compares 7 harnesses on 200 hard tasks
Harness architecture, not model choice, drives the spread in this evaluation — a ten-point gap on identical model and tasks. Teams building or buying browser agents should evaluate their orchestration stack separately from model selection. The benchmark's open CMU-derived task set gives developers a reproducible yardstick for comparing agent frameworks like Codex-based setups against Browser Use 1.0.
Browser Use released Benchmark v2.1, measuring 7 browser harnesses on 200 very hard, long-horizon tasks while holding the model constant: GPT 6.1 Terra Medium, the same tasks, and the same judge, so only the harness differed. The tasks mostly come from the Odysseys v2 collaboration with CMU, plus a few additional tasks on top.
The results show that harness choice alone moved scores widely: six browser tools running inside Codex, the top harness on FrontierHarness, scored between 62.4 and 73.5, while the Browser Use 1.0 agent reached 77.5 on the same model and workload.
The finding matters because it isolates harness design as a decisive variable in agent performance, independent of model quality. Teams deploying browser agents should benchmark their orchestration layer, not just swap models, since the gap between the best and weakest harness on identical tasks exceeds ten points.
We measured 7 browser harnesses on 200 very hard tasks same model (gpt 6.1 terra medium), same tasks, same judge. only the harness changes > Browser Use Benchmark v2.1: 200 super hard long horizon tasks, mostly from our Odysseys v2 collab with CMU + a few extra on top > 6 browser tools inside codex (top harness on frontierharness): 62.4 to 73.5 > browser use 1.0 agent: 77.5📊 10 ⚡
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include browser use, Codex, agents, benchmarks.
Source attribution: curated from Browser Use (@browser_use).
