EarthVerse基准测试为评估科学智能体提供了全面的方法，揭示了当前智能体在跨证据和尺度上的不足，值得专业人士关注。
EarthVerse基准测试通过405个可复现任务评估科学智能体，基于199个记录事件和19种灾害类型。评估25个模型和智能体系统，平均答案单元准确率为84.65%，最高Strict@95为34.81%。测试显示当前智能体常在单个步骤完成，但缺乏跨证据、尺度、单元、计算和物理解释的一致性链。EarthVerse为测量动态地球系统中端到端科学可靠性提供可复现基础。
EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazards
Earth-system analysis reconstructs changing physical processes from observations that differ in source, scale, timing, and modality. Natural hazards make this work consequential because incomplete evidence can change estimates of severity, exposure, and mechanism. We introduce EarthVerse, a benchmark that evaluates scientific agents through package-scoped investigations. Its 405 reproducible tasks are grounded in 199 documented events and 19 hazard families. Agents inspect heterogeneous event packages, choose compatible evidence, execute transparent calculations, reconcile source differences, and preserve provenance in the final answer. We provide executable ground truth that decomposes each task into fine-grained answer units, together with task-specific rubrics that assess the supporting research process while allowing multiple valid paths. We evaluate 25 model and agent systems under a controlled tool-using protocol, then use controlled studies to locate failures in evidence access, tool selection, memory, reasoning, interaction, and scientific execution. Across systems, the best mean answer-unit accuracy is 84.65%, while the highest Strict@95 is only 34.81%. The gap shows that current agents often complete individual steps without maintaining a consistent chain across evidence, scales, units, calculations, and physical interpretation. EarthVerse provides a reproducible basis for measuring end-to-end scientific reliability in dynamic Earth systems.