Papers

研究:LLM 智能体在零样本协调任务中表现不佳

Zero-Shot Coordination among LLM Agents

Why we picked it

一篇研究智能体协作短板的论文:前沿 LLM 在 Hanabi 这类任务里互相配合经常失败,作者提出 CFD 方案还能自动生成,代码已开源。

arXiv 论文《Zero-Shot Coordination among LLM Agents》研究了独立开发的 LLM 智能体在测试时如何协作。作者用 Hanabi 等环境对比发现,前沿 LLM 智能体即使在使用简单通用脚手架的极简环境中也难以协调,主要原因是缺乏对协调问题本身的理解以及对伙伴信念的推理能力偏弱。论文提出 Coordination-friendly definitions(CFDs),并证明 CFD 可以自动发现,无需人工设计,代码已在 GitHub 开源。

Original · arXiv cs.LG

Zero-Shot Coordination among LLM Agents

We study zero-shot coordination (ZSC), where independently developed agents must coordinate at test time. While ZSC has been well studied in the RL literature, far less is known about the performance of LLM agents despite their increasing deployment in such settings. Existing work on LLM coordination often relies on specialised scaffolding that independently developed agents are unlikely to share in practice. Moreover, evaluations on complex environments (e.g., Hanabi) make it difficult to pinpoint the sources of coordination failure. In contrast, we focus on simple, general-purpose scaffolds in minimal environments designed to isolate specific coordination challenges. Our results show that even in these controlled settings, frontier LLM agents struggle to coordinate, largely due to limited understanding of the coordination problem and weak reasoning about their partner's beliefs. While LLMs introduce semantic information as an additional axis for coordination, they nonetheless fail to exploit this structure effectively. Towards this, we propose Coordination-friendly definitions (CFDs) as a principled approach for enabling robust coordination among LLM agents. Finally, we show that CFDs can be discovered automatically, removing the need for manual engineering. Code is available at https://github.com/ahayler/zsc-llm-agents.