斯坦福团队推出NashDreamer，解决零和博弈中的非平稳性问题，比无模型基线训练效率更高。
NashDreamer是一种基于模型的强化学习框架，专为双人零和博弈设计。该框架引入了多智能体循环状态空间模型(MARSSM)，在四个基准游戏中显著提升了早期训练的样本效率。研究还分析了Dreamer算法族在随机环境中面临的后验崩溃问题。
NashDreamer: Model-Based Reinforcement Learning for Zero-Sum Imperfect-Information Games
Model-based reinforcement learning (MBRL) has achieved remarkable results in single-agent domains, yet its extension to competitive imperfect information games (IIGs) remains underexplored. In multi-agent settings, opponent-induced non-stationarity complicates the learning process, and decentralized model learning faces severe identifiability barriers, which we argue make centralized model learning a mathematical necessity. Building on this analysis, we propose NashDreamer, a principled MBRL framework for two-player zero-sum IIGs. It introduces a centralized Multi-Agent Recurrent State-Space Model (MARSSM) that decouples environment dynamics from the effect of players' strategies on their individual observations. NashDreamer is designed to use arbitrary policy gradient algorithms and inherits their convergence guarantees towards Nash equilibria under an idealized model. Empirical evaluations across four benchmark games demonstrate that NashDreamer substantially improves sample efficiency over model-free baselines early in the training. Finally, we theoretically analyze the architecture's optimization landscape, identifying the vulnerability of the Dreamer family of algorithms to posterior collapse in stochastic environments. We leave it as an open challenge.