TT-MBRL：用测试时梯度下降将记忆编码进世界模型
Encoding Memory in World Models with Test-Time Model-Based Reinforcement Learning
强化学习老难题——长期记忆，这篇论文在测试时用梯度下降把记忆写进世界模型，Memory Maze 上刷新 SOTA，做 MBRL 的值得看看方法细节。
论文提出 Test-Time Model-Based Reinforcement Learning（TT-MBRL），在测试阶段通过梯度下降把单局观测的记忆编码进世界模型参数，使每局学到独立的世界模型，区别于依赖上下文记忆的传统做法。作者先在简化版 Memory Maze 2D 环境验证概念，在不同世界模型架构下均取得一致提升。在 3D Memory Maze 基准上，该方法刷新了 state-of-the-art 成绩。同时 TT-MBRL 在 DeepMind Control 和 Atari 100k 等非记忆类基准上保持了强表现。
Encoding Memory in World Models with Test-Time Model-Based Reinforcement Learning
Model-based reinforcement learning (MBRL) approaches have demonstrated strong performance across diverse domains in recent years, learning accurate world models from agent trajectories. However, despite this progress, current approaches still struggle to surpass human performance on tasks that require long-term memory and planning. In this work, we address the problem of long-term memory in MBRL by moving beyond the commonly studied paradigm of architecture design. We introduce Test-Time Model-Based Reinforcement Learning (TT-MBRL), using gradient descent at test time to encode the memory of episode observations into world model parameters. This enables the agent to learn a distinct world model for each episode, improving memory capabilities compared to traditional approaches that solely rely on contextual memory. We first establish a proof of concept on a simplified two-dimensional variant of the Memory Maze environment, which we refer to as Memory Maze 2D, showing consistent improvements across different world model architectures. We then demonstrate the effectiveness of TT-MBRL in the 3D domain, achieving new state-of-the-art results on the Memory Maze benchmark. Finally, we show that TT-MBRL maintains strong performance on non-memory benchmarks such as DeepMind Control and Atari 100k.