Anthropic 量化智能体编程评测中的基础设施噪声
Quantifying infrastructure noise in agentic coding evals Feb 05, 2026
做 AI 编程评测的团队终于有了量化噪声的方法论——基础设施差异能让结果偏差 30%，建议所有做 agentic coding 评估的开发者点开，避免被虚假分数误导。
Anthropic 发布了一项研究，量化了基础设施噪声对智能体编程评测的影响。他们发现，不同的运行环境、工具链版本和硬件配置会导致评测结果出现显著偏差，最高可达 30%。
这项研究提出了标准化评测流程的建议，帮助开发者更准确地评估 AI 编程助手的真实能力。对于依赖评测结果进行模型选型和优化的团队，这直接关系到决策的可靠性。
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include 智能体, 编程助手, 评测, 基础设施噪声.
Primary source verification: published directly by Anthropic: Engineering with first-party authority. Cross-source corroboration: 7 independent sources have covered this development (including arXiv: Anthropic, The Rundown AI, Claude: Blog), confirming industry-wide signal strength.
