Reflex解决了强化学习样本效率低下的痛点，尤其适合做连续控制任务的RL研究者——直接集成PPO/SAC就能提升性能，值得一试。
强化学习长期面临样本效率低下的问题，利用群不变马尔可夫决策过程（G-invariant MDPs）是一种有前景的缓解方法。现有工作主要关注基于图像的强化学习和旋转对称性（如SO(2)），而基于状态的强化学习和反射对称性尚未充分探索。本文提出Reflex范式，专注于基于状态的连续控制任务，利用反射对称性（包括轴向反射和双侧反射），并与PPO和SAC等算法无缝集成。通过理论分析对称性保持的最优值函数和策略，Reflex引入对称性正则化机制，在OpenAI Gym和DeepMind Control基准测试中显著提升了样本效率和性能。代码已开源。
Reflex: Reinforcement Learning with Reflection Symmetry Exploitation in State-Based Continuous Control
Reinforcement learning has long struggled with poor sample efficiency. One promising approach to mitigate this problem is leveraging group-invariant Markov Decision Processes ($G$-invariant MDPs). Existing works in this direction have primarily focused on image-based RL and rotational symmetry such as $\mathrm{SO(2)}$, leaving state-based RL and reflection symmetry largely underexplored. In this work, we focus on state-based continuous control tasks and exploit reflection symmetry by introducing Reflex, a paradigm that seamlessly integrates with both on-policy and off-policy RL algorithms. We formalize two types of reflection-axial reflection and bilateral reflection, and characterize their corresponding transformations. Building on a theoretical analysis of symmetry-preserving optimal value functions and policies, Reflex integrates reflection symmetry into policy learning through principled symmetry regularization mechanisms. We integrate Reflex with PPO and SAC, and evaluate it on a suite of OpenAI Gym and DeepMind Control benchmarks, demonstrating superior performance over standard baselines while improving sample efficiency. Our code is available at https://github.com/TonyStark042/Reflex.