这篇论文提出了一种简单却有效的IER机制,能让强化学习智能体通过重复成功动作更快学习。
研究人员提出即时回合重复(IER)机制,通过立即重复成功回合中的动作序列来提升强化学习样本效率。该机制被集成到SAC和TD3算法中,并在MuJoCo和DeepMind Control Suite等基准测试上进行了验证。实验结果表明,这种简单方法优于标准基线和自我模仿学习基线,在机器人操作任务中也表现出色。
Repetition as Reinforcement: Enhancing Sample Efficiency via Instant Episode Repetition in Reinforcement Learning
Repetition is a fundamental mechanism in human learning, where revisiting successful experiences strengthens memory, consolidates skills, and improves future performance. Motivated by this biological principle, we introduce Instant Episode Repetition (IER), a simple and novel mechanism that improves sample efficiency by immediately repeating action sequences from successful episodes during environment interaction. Unlike conventional approaches such as Experience Replay and Self-Imitation Learning (SIL), which passively reuse past experience during training updates, IER directly influences the data collection process. Upon identifying a high-reward episode, the agent repeats its action sequence for a fixed number of subsequent episodes, reinforcing valuable behaviors through renewed interaction with the environment. We integrate IER into state-of-the-art SAC and TD3 algorithms and evaluate its effectiveness on continuous-control benchmarks, including MuJoCo, the DeepMind Control Suite, and a real-world dynamic object translation task with a robotic manipulator. Experimental results demonstrate that this simple mechanism improves learning performance over standard and self-imitation-based baselines.