Forecasters Missed AI Milestones Like IMO Gold by Years
The pattern of systematic underestimation on benchmarks while overestimating biological risk offers a useful correction to how decision-makers calibrate AI timelines. Researchers, investors and policy teams can use these deltas to adjust their own probability estimates rather than relying on expert medians. Superforecasters being more skeptical than domain experts is itself a finding that challenges the assumption that forecasting skill transfers to technical fields.
A retrospective comparing AI capability forecasts against actual outcomes shows expert and superforecaster predictions were mostly too conservative.
IMO gold was achieved in July 2025, five years before experts' median estimate of 2030, and 90% on Cybench arrived in 2026 versus forecasts of 2028-2030.
MATH (87.8%) and MMLU (88.
7%) scores reached mid-2024 levels that 2022 forecasts gave only 9-25% probability.On biorisk, the opposite occurred: it resolved at 5% (RCT), far below superforecasters' 16% and virologists' 40%.
OpenAI's September 2026 claim of solving Navier-Stokes, targeting verification by end of 2027, remains unconfirmed.
Forecasters were mostly wrong about AI.
On capabilities
- Claimed Millennium Prize problem solution (September 2026 claim by OpenAI re Navier–Stokes, targeting end of 2027, awaiting verification from prize organization; forecast: 10% [experts] and 5% [SF]) - IMO gold (July 2025, forecast: 2030 [experts] and 2035 [SF]), virology troubleshooting task comparable to a strong team of humans (April 2025, forecast: 2030 [experts] and 2034 [SF]) - 90% on Cybench (2026; forecast: 2028 [experts] and 2030 [SF]) - MATH score of 87.8% and MMLU score of 88.7% (mid-2024, forecast: 21%/25% probability [experts] and 9%/7% [SF] as of 2022) - QuALITY hard-subset score of 69.3% (2024 target met a year early; forecast: 44% probability [experts] and 20% [SF])
The biggest misses were IMO gold, which occurred five years before experts’ median estimate, biorisk (which resolved much lower, at 5% (RCT), than expected by SFs (16%), biosecurity experts (23%), and virologists (40%)), and Navier-Stokes, which is not yet officially confirmed.
The biggest misses in terms of percentage points have been LiveCodeBench Pro Hard (actual: 53.8% in May 2026; forecast: 12%-14% by the end of 2026) and FrontierMath Tiers 1-3 unadjusted (actual: 40.7% as of late 2025; forecast: around 30%-31%). Superforecasters were more skeptical than exper ts on the timing of technical achievements.
SF = Superforecasters
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include AI benchmarks, forecasting, reasoning models, AI safety.
Source attribution: curated from @koltregaskes (@koltregaskes). Cross-source corroboration: 5 independent sources have covered this development (including Artificial Analysis, OpenAI Blog, arXiv: OpenAI), confirming industry-wide signal strength.