统一约束视角下的鲁棒强化学习正则化方法研究
A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning
这是篇关于强化学习正则化方法的论文，作者提出了新方法，通过理论分析和实验验证了其有效性。
这篇论文提出了一种新的鲁棒强化学习正则化方法，通过推导新的性能差距上界来统一现有方法。该方法将KL散度惩罚纳入正则化目标，并构建了约束优化问题来更新正则化权重。实验在多个连续控制任务上验证了其有效性。
A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning
Regularization-based methods have become a standard approach for training Deep Reinforcement Learning policies against adversarial input perturbations. In this paper, we unify these methods by deriving new upper bounds on the performance gap between the nominal and worst-case policies. Each upper bound is expressed as an existing regularization objective plus a KL-divergence penalty between the nominal and worst-case policies, which further explains why adding a KL penalty improves robustness in practice. Building on these bounds, we formulate robust training as a constrained optimization problem, showing that existing methods correspond to the special case of a fixed Lagrange multiplier. We instead update the multiplier jointly with the policy to automatically tune the regularization weight. Finally, we conduct extensive adversarial evaluations across several continuous control tasks to validate our theoretical analysis.
