Papers

TradeWar 基准发布:用经济结算评估 LLM 谈判能力

TradeWar: A Benchmark for Execution-Grounded Trade Negotiation with Adaptive LLM Agents

Why we picked it

研究者做了个贸易谈判模拟基准,LLM 光会签协议没用,得看结算后是否真创造价值,TACTIC 智能体靠复盘越谈越好。

arXiv 论文提出 TradeWar,一个多国贸易谈判模拟基准,将谈判对话转化为具体关税与贸易份额政策,并通过经济健康指数(EHI)结算评估。配套提出的 TACTIC 智能体不更新模型权重,仅靠跨轮次经验与对手行为分析迭代谈判策略,EHI 从 58.2 提升至 73.2。对照之下,仅用零样本 CoT 的 PromptLLM 虽达成 75% 协议率,最终 EHI 只有 57.1。在技术管制与能源危机冲击测试中,TACTIC 的性能降幅也小于 PromptLLM 和 MemoryLLM。

Original · arXiv cs.AI

TradeWar: A Benchmark for Execution-Grounded Trade Negotiation with Adaptive LLM Agents

Multi-turn negotiation remains a challenging problem for large language model (LLM) agents. While LLMs can conduct natural-language dialogue, existing benchmarks predominantly assess only agreement rates, overlooking whether negotiated commitments actually create value once executed. To address this gap, we introduce TradeWar, a multi-country simulation benchmark that translates negotiation dialogues into concrete tariff and trade-share policies, enforces capacity and network constraints during settlement, and evaluates outcomes via an Economic Health Index (EHI). To solve this execution-grounded task, we propose TACTIC (Trade-aware Adaptive Consolidation and Theory-of-mind for International Commerce), an agent that iteratively refines its negotiation strategy across episodes-without updating base LLM weights-by leveraging prior experience, opponent behavior, and past settlement outcomes. Despite using zero-shot chain-of-thought (CoT) prompting, PromptLLM attains a 75% agreement rate but only 57.1 final EHI (*100), whereas TACTIC reaches 68.8. Across 20 sequential episodes and three seeds, TACTIC's final EHI improves from 58.2 to 73.2, compared with 65.2 for an ablation without cross-episode updates. Under technology-controls and energy-crisis shocks, TACTIC also exhibits smaller performance degradation than PromptLLM and MemoryLLM. These results indicate that CoT deliberation and interaction memory alone do not fully address delayed execution feedback, whereas settlement-grounded adaptation improves long-horizon negotiation in TradeWar.