Agent Arena 分析 20840 条轨迹：编码智能体差评首因是代码跑不通
Arena 拆了 20840 条编码智能体反馈：69% 差评是代码跑不通，Fable 5.1 比 Astra 更少被吐槽。
Arena 团队分析了 Agent Arena: Code 平台上 29 个模型的 20840 条用户反馈轨迹，用直接的正负评价来衡量编码智能体的实际表现。
投诉中 69.1% 指向代码无法运行，其后是输出不完整（27.
1%）、收尾与可用性差（27.1%）和忽视指令（20.
3%）。Fable 5.
1 的“slop”与设计类投诉占 3.9%，低于 Astra 的 5.
7%，“flaky”类投诉占 3.1%，也低于 Astra 的 5.
0%。29 个模型的数据还显示，各实验室的新一代前沿模型获得的正反馈占比普遍更高。
What can user praise and complaints tell us about coding agents themselves? We analyzed 20,840 traces from Agent Arena: Code across 29 models, and found that direct feedback unlocks novel opportunities in tracking the frontier. Some topline findings: - Today’s frontier models receive markedly more positive feedback. Overall, newer models across labs tend to show a more positive feedback balance. - Broken code is still the biggest driver of complaints, sloppy behavior comes second. 69.1% of complaints point to code not working. The next themes are incomplete output (27.1%), weak finish and usability (27.1%), and ignored instructions (20.3%). - Models share common weaknesses, but differ in how they disappoint. Fable 5.1 attracts fewer “slop” and design complaints than Astra (3.9% versus 5.7% of sampled traces), and fewer complaints about being “flaky” (3.1% versus 5.0%). Read more by diving into the full article from @DawidGalarowicz below. Arena.ai @arena x.com/i/article/2101… 🔗 View Quoted Tweet📊 7 ⚡
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Agent Arena, Fable 5.1, 编程助手, 智能体.
Source attribution: curated from lmarena.ai (@lmarena_ai). Cross-source corroboration: 11 independent sources have covered this development (including 宝玉, AI Will, eric zakariasson), confirming industry-wide signal strength.
