做智能体开发和评测的团队终于有了一个贴近真实工作场景的排行榜——Agent Arena 用 30 万+ 任务和 200 万+ 工具调用数据，告诉你哪个模型在写代码、做研究、处理文档时真正靠谱，值得点开看看你的模型排第几。
Agent Arena 发布了一个全新的智能体排行榜，不再依赖传统基准测试中的孤立问题，而是评估 AI 模型在真实用户任务中的表现，包括编写代码、构建应用、研究主题、创建文档和分析文件等。该排行榜基于 30 万+ 任务、200 万+ 工具调用和 4000 万行代码的数据，综合考量任务成功、可引导性、错误恢复、用户表扬/投诉和工具幻觉等信号。结果显示，GPT-5.5 High 以 +10.7% 的净改进率领先，Claude Opus 4.7 Thinking 和 GPT-5.4 High 紧随其后。该排行榜的核心价值在于将智能体视为工作系统，综合评估模型选择、工具使用、恢复行为和用户满意度。
Arena just released a real-world agent leaderboard…
Arena just released a real-world agent leaderboard that ranks AI models by how well they complete actual user jobs, not isolated benchmark questions.
The system tracks agents using web search, files, and terminal tools while people ask them to write code, build apps, research topics, create documents, and analyze files.
The problem with almost all traditional AI benchmarks is that they test clean tasks, while agents now handle messy work like coding, research, documents, web browsing, files, and terminal commands.
Agent Arena tries to measure agents inside real work sessions, where users correct them, approve results, complain, download files, and expose tool failures as the task unfolds.
Its core idea is to treat each model choice like a test condition, then estimate how much that model improves task outcomes compared with a baseline.
The leaderboard combines 5 signals: confirmed task success, praise versus complaint, ability to follow corrections, recovery from terminal errors, and whether the agent invents tools that do not exist.
The data is large enough to show real behavior patterns, with 300K+ tasks, 2M+ tool calls, and 40M lines of code produced by agents.
The score combines task success, steerability, bash recovery, praise vs. complaint, and tool hallucination, which means the model is judged by whether it finishes, recovers, accepts correction, and avoids fake tool calls.
GPT-5.5 High leads with +10.7% net improvement, followed by Claude Opus 4.7 Thinking at +9.5% and GPT-5.4 High at +8.9%.
The most useful detail is that agents fail like workers under pressure: they can leave one part incomplete, claim the job is done, or sound confident while backing down after correction.
Arena’s strongest contribution is treating agents as working systems, where model choice, tool use, recovery behavior, and user satisfaction all count together.
- lmarena.ai06-05 14:19原文