CritPt v2 Benchmark Launches with Agentic Code Execution
The 54-point gap between GPT-5 and GPT-6 Astra in about a year shows how quickly frontier models are absorbing research-level physics when given code execution. Tracking these scores before the late October Intelligence Index update helps researchers and benchmark watchers pick models for scientific workflows. Compare against Claude Opus 5.5 at 78% when choosing a frontier model for agentic computation tasks.
Artificial Analysis has launched CritPt v2, a revised version of the research-level physics benchmark containing all 70 challenges, updated with the CritPt authors to fix errors, underspecified statements and grader instability. Models now run agentically, writing and executing code in an offline sandbox. GPT-6 Astra (max) leads at 83%, followed by GPT-6.1 Sol (max) at 82% and Claude Opus 5.5 (max with fallback) at 78%. GPT-5 (high), released in August 2025, scores 29%, 54 points below GPT-6 Astra. Artificial Analysis will fold CritPt changes into its Intelligence Index in late October.
CritPt v2 has launched on Artificial Analysis: the CritPt authors have revised all 70 research-level physics challenges, and models now solve them with a code-execution tool
Following feedback from the research community, the CritPt team and physics expert collaborators reviewed every challenge, corrected underspecified statements, errors, and typos, and improved the stability of the grader. We now run CritPt v2 agentically: models can write and run code in an offline sandbox with tools for computation.
We’ll be updating the Intelligence Index in late October, including changes to CritPt.
Key takeaways:
➤ Recent OpenAI releases top the leaderboard: GPT-6 Astra (max) and GPT-6.1 Sol (max) score highest at 83% and 82%, followed by Claude Opus 5.5 (max with fallback) at 78% and GPT-5.6 Sol (max) at 75%
➤ CritPt v2 shows the rapid progress at the frontier on research-level physics: GPT-5 (high), released in August 2025, scores 29% on CritPt v2, 54 points below GPT-6 Astra (max)
Thank you to @MiniHui_zhu, @MinyangTian1, @haopeng_uiuc and the broader CritPt team for their work on this revision and their continued collaboration
