DFT-Agent Bench 发布：评估 LLM 智能体自主执行密度泛函计算能力
DFT-Agent Bench: Scaffold Effects and Execution Provenance in Autonomous DFT
搞第一性原理计算的值得看看：200 多道 Quantum ESPRESSO 任务测 LLM 智能体，还带执行溯源审计防止“没真跑也拿高分”。DeepSeek 和 Opus 在弹性张量上表现差距明显，脚手架的作用因模型而异。做科研智能体的人可以拿来当评测模板。
DFT-Agent Bench 是一个包含 200+ Quantum ESPRESSO 任务的基准，覆盖弛豫、电子结构、状态方程和弹性张量四类计算流程，每个任务带有 QE 导出的 ground truth 和可交叉验证的公开参考文献。基准的独特之处在于除结果评分外，还引入了 provenance 审计，检查智能体是否真正满足了 QE 执行契约，防止未实际运行计算却获得高分的虚假执行。
在统一执行框架下，研究者将接近裸配置的 ReAct 基线与捆绑的 v5.1 脚手架在两个模型上对比。完整脚手架在 bands+DOS 任务上提升 +0.08/+0.11，d-electron vc-relax 提升 +0.17，而弛豫和状态方程任务的高分格子已接近饱和。弹性张量结果呈现模型差异：deepseek-v4-pro 从 0.18 提升到 0.40，claude-opus-4-8 则从 0.62 微降到 0.59。100 任务固定切片（关闭检索）的对照分析显示 DeepSeek 增益 +0.17、Opus 增益 +0.10（p < 0.001），支持总体模式。
provenance 门槛能捕获那些可能拿到高评分但执行不完整或未标记的任务，本质上是对执行契约的检查而非反作弊测试。匿名评审工件包含冻结诊断切片、提示词、评分器和求解器版本，供社区复现。
DFT-Agent Bench: Scaffold Effects and Execution Provenance in Autonomous DFT
Large language models are increasingly deployed as autonomous agents for first-principles materials simulation. We report empirical findings from DFT-Agent Bench, a benchmark of 200+ Quantum ESPRESSO tasks spanning relaxation, electronic structure, equation-of-state, and elastic-tensor pipelines over multiple material tiers. Each task carries a QE-derived ground truth with public-reference cross-checks where available, and a provenance audit checks whether the required QE execution contract was met. Under one execution harness, we compare a near-bare ReAct baseline with a bundled v5.1 scaffold on two models. The complete scaffold improves bands+DOS by +0.08/+0.11 and d-electron vc-relax by +0.17 on both models, while high-scoring relaxation and equation-of-state cells leave little room for improvement. The elastic result is model-dependent (0.18 to 0.40 for deepseek-v4-pro and 0.62 to 0.59 for claude-opus-4-8). A fixed 100-task analysis with retrieval disabled supports the aggregate pattern: the fully frozen DeepSeek comparison gains +0.17, while the Opus comparison gains +0.10 against a historical pool baseline (p < 0.001). The provenance gate catches incomplete or unmarked executions that can receive high rubric scores; it is an execution-contract check, not a test for intentional fraud. These results characterize the evaluated DFT workflows. The anonymous review artifact contains the frozen diagnostic slice, prompts, scorer, and solver revisions: https://anonymous.4open.science/r/231e31re91287h/README.md.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include DFT-Agent Bench, Quantum ESPRESSO, deepseek-v4-pro, claude-opus-4-8.
Primary source verification: published directly by arXiv: DeepSeek with first-party authority.