AgentWorld 基准发布：测试多智能体长程协作能力
AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs
一个新的多智能体协作基准，用 MMORPG 沙盒逼模型跑 50 多轮团队任务，Gemini 3 Flash 也只拿到 52% 成功率，能看清现在模型协作到底差在哪。
AgentWorld 是一个包含 100 个人工标注任务（另有 100 个增强变体）的多智能体协作基准，任务在 MMORPG 沙盒中展开，需 3-20 个角色能力不对齐的智能体跨 50+ 轮交互协调。基准提出 CCE（Causal Collaboration Effectiveness）指标，通过因果依赖图衡量团队行动中有多少真正推动了结果。实测 Gemini 3 Flash、Claude Haiku 4.5、GPT-5 Mini 和 DeepSeek R1-70B，最好成绩也只有 52.0% 任务成功率，失败模式集中在沟通断裂、角色混淆和无法维持共享计划。项目已完全开源。
AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs
Existing multi-agent benchmarks primarily test in competitive settings, short-horizon interactions under 20 steps, or simply aggregate individual performance, failing to isolate and highlight genuine collaboration capabilities of LLM-based agents. We introduce AgentWorld, a benchmark of 100 human-annotated tasks (with 100 augmented variants) for evaluating long-horizon, multi-agent collaboration. Tasks span 50+ interaction rounds across a rich MMORPG sandbox and require 3-20 agents with asymmetric roles and abilities to coordinate through communication, joint planning, and resource sharing under a blackbox setting where each agent acts independently without access to others' internal states. To quantify collaboration effectiveness in addition to conventional binary task success, we propose Causal Collaboration Effectiveness (CCE), a graph-based metric that traces causal dependencies between agent actions and measures what fraction of a team's effort actually contributed to the outcome. Experiments with Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B show that even the best model achieves only 52.0% task success, with systematic failure modes including communication breakdowns, role confusion, and inability to maintain shared plans across rounds. AgentWorld is fully open-source.
