论文精选

DNQ:面向部分可观测多人博弈的深度Nash Q网络

DNQ: Deep Nash Q-Network for Partially Observable n-Player Games

精选理由

做多智能体博弈或竞标系统研究的团队,这篇论文给出了一个实用的均衡监督框架,成对方法在扩展性上表现突出,值得关注其训练成本与策略精度的平衡思路。

AI 摘要

论文提出DNQ框架,用于训练在共享约束、有限信息和重复互动环境中的竞标智能体。该方法通过轨迹收集、基于评论家的收益估计、均衡计算和策略模仿的交替循环,在每一状态利用共享评论家预测收益矩阵,外部求解器计算纳什均衡,并通过KL散度最小化训练智能体。实验对比了成对和精确两种变体,显示成对方法在智能体数量扩展上更具优势,而精确方法在博弈规模增大时计算不可行。该研究揭示了重复竞争环境中策略保真度与可扩展性之间的权衡。

原文 · arXiv cs.LG

DNQ: Deep Nash Q-Network for Partially Observable n-Player Games

Many real-world competitive systems require multiple decision-makers to act simultaneously under shared constraints, limited information, and repeated interaction, as in auctions, resource allocation, and security competition. We study multi-turn simultaneous bidding as a controlled testbed for such problems and propose DNQ, a solver-in-the-loop equilibrium supervision framework for training bidding agents. DNQ alternates between trajectory collection, critic-based payoff estimation, equilibrium computation, and policy imitation. At each visited state, a shared critic predicts either pairwise payoff matrices or an exact N-player payoff tensor, an external solver computes equilibrium strategies, and the agents are trained by minimizing the KL divergence between their masked policies and the solver-derived equilibrium targets. We focus on a scalable pairwise formulation that greatly reduces equilibrium-solving cost and training time compared with the exact formulation, while the shared critic amortizes payoff learning across agents and states. Experiments compare the pairwise and exact variants using critic loss, policy entropy, bidding resource usage, and training cost, showing that the pairwise method scales to larger numbers of agents, whereas the exact method becomes computationally impractical as the joint game grows. These results illustrate the trade-off between strategic fidelity and scalability in repeated competitive environments.