FrontierPhysics基准发布：56个任务评估AI端到端物理研究能力
FrontierPhysics: Evaluating Agents for End-to-End Physics Research
42位物理学家出题考AI做完整科研，22个模型没一个能及格，具体差在哪篇里分析得很清楚，做智能体评测的值得看。
FrontierPhysics是一个端到端物理研究基准，包含56个任务，由42位物理学家基于自己的研究设计，覆盖理论、计算与实验物理九个领域。每个任务要求AI智能体完成文献调研、制定研究计划、开展实验并输出结构化论文。评测结合确定性验证器和专家评分标准，同时检验研究正确性和可发表性。在22个模型、26种配置的测试中，没有配置在168次尝试中通过超过46次。人工分析显示前沿智能体的短板是推理不够严谨和缺乏专家直觉，而不是物理知识欠缺或执行错误。
FrontierPhysics: Evaluating Agents for End-to-End Physics Research
AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
