TeamGym: 评测分布式信息下 AI 智能体团队决策的环境
TeamGym: A Gym for Team Decision-Making by AI Agents with Distributed Information
一个专门测智能体"团队分工"的新环境，8 个模型一测，信息一分散成功率直接掉 22.9 个点，还教你怎么用 RL 训出协作能力。
TeamGym 基于 12 个来自法律、金融、医疗和软件工程的真实语料构建，用于评测和训练语言模型智能体的团队决策能力。其核心方法"pivotal information"会找出对正确决策既必要又联合充分的最小事实集合，从而控制信息在参与者间的分布并追踪智能体的识别与整合能力。在 8 个语言模型的测试中，把关键信息分散给多个参与者后，成功率平均下降 22.9 个百分点，即通信鸿沟；若参与者还需自行从原始材料中提取关键信息，成功率再降 19.9 个百分点，即检索鸿沟。研究团队进一步用团队决策损失做强化学习训练，在两次实验中提升了最终决策质量，并涌现出参与、发言选择和沟通上的协调行为。
TeamGym: A Gym for Team Decision-Making by AI Agents with Distributed Information
Many real-world decisions require teams to integrate information distributed across participants. Yet AI agents are still primarily trained and benchmarked in isolation. We introduce TeamGym, an environment for evaluating and training language model-based agents on team decision-making problems constructed from real-world legal, finance, medical, and software engineering corpora. TeamGym introduces a method to identify a minimal set of facts, each element of which is necessary and all of which are collectively sufficient for a correct final decision, which we call pivotal information. This enables TeamGym to both control the distribution of pivotal information and track how well agents identify and integrate pivotal information when making team decisions. Examining 8 language models on 12 source corpora, we find that current AI agents often fail to effectively share information: Distributing pivotal information across participants lowers success rate by 22.9 percentage points on average relative to giving the same information to a single model (a communication gap). We also document that participants do not effectively retrieve decision-relevant information on their own: A team's success rate is lowered by a further 19.9 points when participants must identify and extract pivotal information from source material rather than receiving it directly (a retrieval gap). To help remedy these deficits, we use TeamGym to train policies through reinforcement learning on team decision loss. Across two policy-training experiments, we find that training improves final-decision quality and induces emergent coordination in participation, speaker selection, and communication, even though the reward evaluates only the final decision.
