Science's Last Exam benchmarks AI research agents on reproducible work
The design borrows Lean's philosophy for AI math — verifiable, tamper-proof grading — and applies it to open-ended science, making contamination and benchmark gaming far harder. Researchers comparing model claims on scientific reasoning should treat SLE's 21.28% figure for DeepSeek V4.1 Flash as a more contamination-resistant signal than conventional Q&A benchmarks. The low score also sets a sober baseline for how far agents remain from genuine discovery.
Science's Last Exam (SLE) is a benchmark led by the US nonprofit AItonomy Foundation that grades AI research agents on re-runnable artifacts. It spans 113 tasks across 9 disciplines, with expert-built checks hidden from the agents during testing. Unlike standard question-answering benchmarks, SLE gives an agent a research goal, data and lab software, then verifies the finished work. Hypotheses are scored against studies published after the model's training cutoff, blocking memorisation. DeepSeek V4.1 Flash led with 21.28% of its hypotheses anticipating later research.
Lean gave AI math a grader nobody can argue with, and empirical science is now getting its own version of that idea.
Science’s Last Exam (SLE), a benchmark, grades research agents on artifacts that can be opened and re-run, with checks that domain experts seal when each task is built.
Led by the US nonprofit AItonomy Foundation, the benchmark spans 113 tasks in 9 disciplines and grades submissions with checks hidden from agents.
Most AI science tests ask questions with known answers; SLE hands an AI agent a research goal, data and lab software, then checks its finished work with tests the agent never sees.
It scores AI hypotheses against studies published after the model’s training cutoff, so memory cannot help. DeepSeek V4.1 Flash led, with 21.28% of its hypotheses anticipating later research.
