做LLM智能体长期部署和持续学习的团队可以关注——MemoPilot用强化学习自动优化记忆策略，比手工调提示更系统，在博弈场景中效果显著，值得在类似任务中尝试。
论文提出MemoPilot，一种插件式记忆副驾驶，通过强化学习显式训练记忆更新过程，使冻结的LLM在连续交互中提升性能。该方法将记忆更新建模为多轮决策问题，采用多轮GRPO端到端优化，引入轮次奖励信号和上下文无关的轮级优势估计，实现更精细的信用分配和稳定训练。在多人石头剪刀布和有限注德州扑克两个测试环境中，MemoPilot的Elo评分分别达到1590和1762，超越所有基线记忆方法和包括DeepSeek-V3.2在内的闭源模型。这项工作解决了现有方法依赖手工设计提示规则、难以对齐记忆更新与长期目标的痛点。
From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory
Large language model (LLM) agents are increasingly deployed in long-running settings where improving through experience at test time becomes important. A common approach is to update an explicit memory after each interaction to guide future decisions. However, most existing methods rely on hand-designed prompting rules, making it difficult to align memory updates with downstream objectives over multi-step horizons consistently. We propose MemoPilot, a plug-in memory copilot that explicitly trains the memory update process to improve a frozen LLM's performance across sequential interactions. We formulate memory updating as a multi-turn decision problem and optimize it end-to-end with multi-turn GRPO. Our training recipe introduces (i) a turn-wise reward signal and (ii) a context-independent, turn-level advantage estimation across rollouts, enabling finer-grained credit assignment and more stable training in multi-turn settings. We evaluate MemoPilot on two testbeds: multi-round Rock-Paper-Scissors (RPS) and Limit Texas Hold'em (LHE). Across both environments, MemoPilot substantially improves test-time learning of a frozen player over strong baselines, ranking first in Elo ratings on both games (1762 on LHE and 1590 on RPS) and outperforming all baseline memory methods and proprietary models, including DeepSeek-V3.2.