做安全强化学习或机器人系统的团队，终于有了一个不用重训模型就能清理毒化数据的方案，值得关注。
离线安全强化学习（Safe RL）依赖静态数据集，容易遭受数据毒化攻击——攻击者注入恶意样本导致策略不安全。本文提出Safe-RULE（安全强化反学习）框架，无需从头重新训练或访问原始训练环境，即可移除毒化数据的影响。该方法在反学习过程中同时考虑任务性能和安全约束，实验表明能有效提升对数据毒化攻击的安全性。
Safe-RULE: Safe Reinforcement UnLEarning
Offline safe reinforcement learning (Safe RL) enables policy learning without online interactions, making it suitable for safety-critical systems such as robotics systems. However, its reliance on static datasets exposes offline Safe RL to data poisoning attacks, where adversaries inject malicious samples that compromise safety and induce unsafe policy behavior. In this work, we propose a new learning paradigm, named safe reinforcement unlearning (Safe-RULE), used as a defense framework to remove the influence of poisoned data without retraining from scratch or requiring access to the original training environment. We further extend reinforcement unlearning to offline Safe RL by explicitly accounting for both task performance and safety constraints during the unlearning process. Experiments across benchmark Safe RL tasks demonstrate that our approach effectively enhances safety performance against data poisoning attacks.