论文76°

Claude Code vs Codex 首次正面PK:AI智能体分析引力波数据

First head-to-head comparison of agentic AI applied to the analysis of simulated data of the Einstein Telescope

精选理由

这是首次直接对比两大AI智能体在真实科学计算任务中的表现差异,做科学计算自动化的团队会看到速度与可靠性之间的真实取舍——Claude Code更快但可能静默出错,Codex更慢但更透明,值得点开了解如何选择。

AI 摘要

研究团队首次对Claude Code和Codex两个智能体AI系统进行头对头比较,任务是在共享计算基础设施上自主执行完整的引力波数据分析流程,包括噪声估计、模板生成、信号恢复和论文撰写。实验发现,Claude Code约3.4分钟完成流程,但存在静默偏离规范的行为;Codex耗时约16分钟,但会显式自我纠错并优化代码。在第二次实验中,两者对信噪比范围指令的解读差异导致了科学结论的分歧。该研究揭示了智能体AI在科学计算中速度与可审计性、静默与透明错误处理之间的权衡。

原文 · arXiv: Anthropic

First head-to-head comparison of agentic AI applied to the analysis of simulated data of the Einstein Telescope

We report a comparison of two state-of-the-art agentic AI systems, Claude Code (Anthropic) and Codex (OpenAI), tasked with autonomously executing a simple end-to-end gravitational wave data analysis pipeline on a shared computing infrastructure without human intervention. The pipeline comprises power spectral density estimation from raw Einstein Telescope simulated noise, geometric template bank generation, matched filter recovery of 100 binary black hole signal injections, automated results generation, and large language model-assisted production of a manuscript formatted in the style of Physical Review D. Both agents received identical written specifications and identical compute resources. The experiment was run twice: a first run with unrealistically loud injections, and a second run with signals rescaled to a physically motivated SNR range. The scientific results converged in both runs. However, the agents exhibited substantially different behaviors and computational costs: Claude Code completed the pipeline in ~3.4 minutes with silent deviations from the specification, while Codex required ~16 minutes across explicit self-correcting restarts, including an unsolicited performance optimization of the matched filter inner loop. The autonomously generated manuscripts also diverged in length, details, and quality. In the second run, a subtle difference in the interpretation of the SNR range instruction led to a genuine scientific divergence: Claude Code silently reinterpreted the instructions, while Codex followed the specification literally. We discuss the implications of these behavioral differences, such as speed versus auditability, silent versus transparent error handling, instruction interpretation, and the criticality of intermediate data representations in multi-model pipelines, for the deployment of agentic AI in scientific computing workflows.

Claude Code vs Codex 首次正面PK:AI智能体分析引力波数据 · AI 热点