做智能体评测或基准测试的团队终于有了一个通用框架——AgentBeats 用智能体评估智能体，解决了传统测试碎片化、难复现的痛点，建议做 Agent 平台或竞赛的开发者点开看看。
AgentBeats 提出了一种全新的智能体评估框架 AAA（Agentified Agent Assessment），由智能体担任裁判，通过 A2A 和 MCP 标准化协议与待测智能体交互，取代传统依赖 LLM 的固定测试框架。该方法统一了评估接口，解决了现有基准测试集成成本高、测试与生产环境不匹配、跨设计公平比较难的问题。研究通过五个月开放竞赛（298 个裁判智能体、467 个参赛智能体）和编程智能体案例验证，证明 AAA 在覆盖度、实用性和保真度上均表现优异。AgentBeats 为智能体评估提供了开放、标准化、可复现的路径。
AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility
Agent systems are advancing quickly across domains, but their evaluation remains fragmented. Most benchmarks rely on fixed, LLM-centric harnesses that require heavy integration, create test-production mismatch, and limit fair comparison across diverse agent designs. The root problem is the lack of an open, agent-agnostic assessment interface. We advocate Agentified Agent Assessment (AAA), where evaluation is performed by judge agents and all participants interact through standardized protocols: A2A for task management and MCP for tool access. Conventional benchmarking defines two separate interfaces, one for the benchmark and one for the agent, while AAA only needs one; this yields a generic, unified framework that separates assessment logic from agent implementation and enables reproducible, interoperable, and multi-agent evaluation. We further introduce AgentBeats as a concrete realization of AAA: we identify five practical operation modes that make standardized assessment compatible with real-world constraints on openness, privacy, and reproducibility. To evaluate our design at scale, we conduct two studies: a five-month open competition that drew 298 judge agents across 12 categories together with 467 subject agents from independent participants, showing that AAA applies across a heterogeneous range of benchmarks; and a case study on coding agents that confirms agentified evaluation preserves fidelity with the public record while surfacing previously missing head-to-head results, yielding research insights about agent design. Combining a community-scale field study and a controlled coding case study, we verify that AAA delivers coverage, practicality, and fidelity across heterogeneous scenarios at scale. Together, AAA and AgentBeats offer a clear path toward open, standardized, and reproducible agent assessment.