PHYRD 基准发布：测试模型复现与扩展理论物理研究的能力
PHYRD: A Living Benchmark for Reproducing and Extending Theoretical Physics Research
物理版科研能力测试题，63 道复现加 43 道发现题，最强模型闭卷也只有七成多正确率，还有合成数据能白嫖涨 11 个点。
arXiv 论文介绍 PHYRD，一个针对理论物理推理的动态基准，包含 63 道 Reproduce 和 43 道 Discover 题。Reproduce 要求推导已发表结果，Discover 要求在修改理论条件后推导新结论，任务由 FormulaForest 系统构建并附带子问题级评分标准。五个模型在 Reproduce 上的整题准确率为 49.21% 到 68.25%，在闭卷 Discover 上为 18.60% 到 74.42%。使用工具后四个模型完成更多 Discover 题，但仍存在概念性错误和论证缺失。FormulaForest 还可用作合成数据，将 Qwen-35B-A3B 在 OpenAI 的 FrontierScience 基准上的准确率从 61% 提升到 72%。
PHYRD: A Living Benchmark for Reproducing and Extending Theoretical Physics Research
Large language models and agents have shown significant power in advanced physical research. In this paper, We introduce PHYRD, a living benchmark for theoretical-physics reasoning with 63 Reproduce and 43 Discover problems. Reproduce tests derivation of published results; Discover tests consequences of modified theoretical conditions. FormulaForest constructs the tasks with reference solutions and subquestion-level rubrics. Across five models, whole-problem accuracy ranges from 49.21% to 68.25% on Reproduce and from 18.60% to 74.42% on closed-book Discover. With tools, four models complete more Discover problems, but conceptual errors and missing arguments persist. Besides, our experimental results demonstrate the value of FormulaForest as synthetic data. On FrontierScience, a benchmark developed by OpenAI, it raises the accuracy of a Qwen-35B-A3B model from 61% to 72%.