DecArena 基准发布：去中心化多智能体协作遇上 10 万工具规模测试
DecArena: Evaluating Large-Scale Decentralized Multi-Agent Collaboration
做智能体系统的人可以看看这个基准：它专门测没有全局编排器、只能靠邻居通信的场景，10 万工具规模下主流 LLM 智能体都翻车，RL 也救不回来。论文给的 GraphHarness 基线涨了 48.5 个点还省成本，思路对搭建大规模分布式智能体很有参考价值。
arXiv 论文 2611.15521 提出 DecArena，一个评估去中心化多智能体系统（MAS）的基准，核心场景是图约束通信下的分布式工具调用，工具规模超过 10 万个。在该设定中，每个智能体只能观测本地状态、只能与相邻智能体交互，没有全局编排器，必须主动发现非本地的工具和技能并与其他智能体协作。
实验结果显示，现有 LLM 智能体在去中心化环境中难以发现非本地相关资源，而标准强化学习训练无法稳定克服这一限制，说明即使使用 SOTA 模型，去中心化 MAS 仍需要更显式的 harness 机制。
作者随后提出基线方法 GraphHarness，将拓扑感知预热与基于 LLM 的自改进路由结合。结果显示，相比基线，GraphHarness 将任务成功率提升 48.5，协作质量提升 30.6，同时成本降低 34.0。
DecArena: Evaluating Large-Scale Decentralized Multi-Agent Collaboration
Emerging personal AI agents raise fundamental scalability and privacy concerns for centralized multi-agent systems (MASs), which rely on a global orchestrator with broad access to agents, resources, and communication. In decentralized environments, global visibility is unavailable. Agents need to proactively discover non-local resources (e.g. tools, skills), and collaborate with other agents to solve tasks. We study decentralized multi-agent collaboration under graph-constrained communication, a natural reflection of human structures. Each agent observes only its local state and can interact only with neighboring agents. We introduce a benchmark with synthetic scalable distributed tool calling with 100K+ tools. Our experiments reveal that recent LLM agents struggle to discover non-local relevant resources and collaborate in decentralized setting. Moreover, standard reinforcement learning fails to reliably overcome this limitation, suggesting that decentralized MAS needs more explicit harness mechanisms for effective collaboration, even using SOTA LLMs. We then present a strong baseline \textbf{GraphHarness}, combining topology-aware warmup with self-improving LLM-based routing. Results show that GraphHarness substantially improves task success (+48.5) and produces higher quality collaboration (+30.6), while using fewer costs (+34.0) than baselines.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include 多智能体, 智能体, GraphHarness, 去中心化协作.
Source attribution: curated from arXiv cs.AI (@Jiali Cheng, Oleg Rokhlenko, Joo-Kyung Kim, Hadi Amiri, Xingzhi Guo).
