这篇论文为多智能体强化学习中的均衡选择问题提供了理论解释和实用机制，做多智能体系统或博弈论应用的开发者值得关注，尤其是对合作均衡有需求的团队可以看看如何通过对手感知修正引导策略收敛。
该论文研究了多智能体策略梯度方法在多个稳定纳什均衡中如何选择的问题。作者提出通过“盆地进入概率”来分析均衡选择，并发现“同伴学习修正”是关键的均衡选择机制：在局部对齐条件下，该修正能增加进入目标稳定纳什集（如帕累托优势均衡）吸引盆地的概率。论文证明了有限展开的Meta-MAPG更新可分解为普通策略梯度加上自身学习和同伴学习修正，且同伴学习修正会改变原始游戏的零更新点。通过退火修正，可在进入盆地后恢复普通策略梯度动力学，并继承局部稳定纳什收敛保证。实验在Stag Hunt、迭代囚徒困境和初步的神经策略协调环境中验证了该观点，显示对手感知更新能增加进入合作盆地的概率。
Equilibrium Selection in Multi-Agent Policy Gradients via Opponent-Aware Basin Entry
Multi-agent policy-gradient methods have been shown to converge locally near stable Nash equilibria. Local convergence, however, does not determine which equilibrium is reached. We study this question through basin-entry probability with respect to a target set of equilibria selected by an external criterion, such as payoff dominance. For finite-unroll Meta-MAPG, we show that the update decomposes into ordinary policy gradient plus own-learning and peer-learning corrections, with controlled sampling noise and finite-unroll bias. We identify the peer-learning correction as the main equilibrium-selection mechanism: under a local alignment condition, the probability of entering the certified attraction region of the target stable-Nash set increases, relative to ordinary policy gradient. Because persistent correction may shift zero-update points of the original game, annealing the correction after entering the basin recovers ordinary policy-gradient dynamics and inherits local stable-Nash convergence guarantees. Experiments in Stag Hunt, iterated Prisoner's Dilemma, and preliminary neural-policy coordination environments support this basin-entry view, showing increased entry into cooperative basins under peer-aware updates.