Ethan Mollick argues autonomous AI benchmarks underestimate economic value
The split Mollick draws matters for anyone setting expectations from AI capability claims: benchmark results on frontier research tasks say little about returns on ordinary, well-specified work. Teams evaluating coding agents should benchmark against their own task mix rather than lab-centric autonomy metrics, and expect human-in-the-loop setups to capture most of the productivity gains. The cited OpenAI internal estimate of a one-month-to-three-months speedup offers a concrete anchor for ROI modelling.