FrontierCode 戳破了现有基准的泡沫，真正衡量代码可维护性而非通过测试——做 AI 编程工具或智能体开发的团队，建议看看这个新标尺，它可能改变你评估模型的方式。
METR 发布 FrontierCode 基准测试，发现超过一半的 SWEBench 结果是无法合并的劣质代码。FrontierCode 包含 1000+ 小时维护者验证的软件工程任务，并设有 3000+ 条评分标准，涵盖代码质量和反作弊机制。最难的 FC Diamond 级别中，Opus 4.8 得分仅为 13.8%。该基准将 AI 编程划分为三个时代：2021 年自动补全（HumanEval）、2023 年通过测试（SWEBench）、2026 年可维护代码（FrontierCode）。数据显示，2025 年底模型能力出现跃升，最易任务的通过率在 4 个月内从 41% 提升至 74%，标志着 AI 编程从 2 次重试 95% 成功率到 6 次重试的质变。
It's finally out!!! @METR_Evals found that more th…
It's finally out!!! @METR_Evals found that more than half of SWEBench results is unmergeable slop. FrontierCode represents over 1000+ hours of maintainer validated software engineering work most frontier models cannot yet solve, much less solve with high quality.
Cog had IOI Gold medalists and top code maintainers Look At The Data — FrontierCode includes 3000+ rubrics covering code quality and anticheat reward hacking plaguing other benchmarks.
FC Diamond is so hard that Opus 4.8 scores 13.8%.
Three eras of AI coding : Three eras of benchmarks
2021 • Autocomplete : HumanEval 2023 • Passing Tests: SWEBench, TerminalBench 2026 • Maintainable Code: FrontierCode
to me the most beautiful chart when I requested a special historical run into all extant old models, the data was finding that the easiest third of FC tasks (in FC Extended) were rapidlly and suddenly solved over late 2025 - Opus almost doubled from a 41% pass rate to 74% in 4 months.
This describes the "WTF happened in Dec 2025" vibe shift that a lot of folks from @dhh to @karpathy have called out: it is the difference between getting 95% success in 2 rerolls vs 6, making it finally feasible to go up the next layer of abstraction in agentic coding, eg @GeoffreyHuntley's ralph loops or @bcherny's /goals or @steipete's "loops that prompt your agents" without fearing too much that things go off the rails.
My guess: as AI accelerates from here, each FrontierCode tier will saturate in sequence, hopefully ~annually. I've already asked the team to prepare FrontierCode 2027....
The old mountains will be destroyed. Their rubble becomes regolith. And from that regolith, the next model forest grows. Circle of life.