AutoLab 填补了现有基准只测短周期任务的空白，做 AI 智能体研究和开发的团队可以直接用它来测试模型的持久迭代能力，值得关注。
AutoLab 是一个新基准，用于评估 AI 模型在超长周期闭环优化任务中的表现，涵盖系统优化、谜题挑战、模型开发和 CUDA 内核优化四个领域。每个任务从次优基线开始，要求智能体在严格时间预算内通过反复实验和反馈改进。测试 17 个前沿模型后发现，成功的关键不是初始尝试的质量，而是持续迭代和利用经验反馈的能力。Claude Opus 4.6 表现突出，但多数模型过早终止或进展有限。该基准开源，旨在推动长周期自主智能体的研究。
AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?
Scientific and engineering progress is fundamentally a long-horizon iterative process: proposing changes, running experiments, measuring outcomes, and continuously refining artifacts. Yet existing benchmarks for frontier models primarily evaluate either single-turn responses or short-horizon agent trajectories, failing to capture the challenges of sustained iterative improvement over extended time horizons. To address this gap, we introduce AutoLab, a new benchmark for ultra long-horizon closed-loop optimization. AutoLab consists of 36 realistic, expert-curated tasks spanning four diverse domains: system optimization, puzzle & challenge, model development, and CUDA kernel optimization. Each task begins with a correct but deliberately suboptimal baseline and challenges agents to improve it within a strict wall-clock budget. Evaluating 17 state-of-the-art models reveals the dominant predictor of success is not the quality of an agent's initial attempt, but its persistence in repeatedly benchmarking, editing, and incorporating empirical feedback. While claude-opus-4.6 exhibits strong long-horizon optimization capabilities, most frontier models, including several proprietary ones, either terminate prematurely or exhaust their budgets with minimal progress. These results underscore the importance of time awareness and persistent iteration in autonomous agents. We open-source the full benchmark, evaluation harness, and task artifacts, to accelerate research toward truly capable long-horizon agents.