ATACOM-DC改进了安全强化学习，通过方向性约束只在必要时限制动作，比普通方法探索更高效。
安全强化学习中，约束常降低学习速度并导致次优性能。ATACOM-DC方法在ATACOM框架基础上引入方向性约束，区分靠近和远离约束边界的动作。仅在必要时激活约束，改善安全与任务性能的权衡。在多个模拟机器人控制任务中评估了约束违规成本和任务性能。代码已开源。
Directional Constraints for Efficient Exploration in Safe Reinforcement Learning
Reinforcement Learning has revolutionized the landscape of robotic research, allowing robust learning of complex robotic skills in simulation. However, real-world deployment in open-ended environments requires strong safety guarantees to prevent dangerous or harmful behaviors. Safe Reinforcement Learning methods address this requirement by enforcing safety constraints. Nevertheless, learning under constraints often reduces learning speed and could lead to suboptimal task performance, as the agent must solve a more complex constrained optimization problem compared to unconstrained settings. To tackle this issue, in this work, we propose an extension of the ATACOM framework, a state-of-the-art reliable safety layer that can be integrated with existing Reinforcement Learning algorithms to enforce constraints derived from prior knowledge of the system or learned directly from data. Our proposed method, named ATACOM Directional Constraints (ATACOM-DC), significantly improves the safety-performance trade-off by introducing directional constraints that distinguish between actions approaching and moving away from constraint boundaries, activating constraint enforcement only when necessary. We evaluate our method across a range of challenging robotic control tasks in simulation, analyzing both constraint-violation costs and achieved task performance. Code and additional material at https://atacom-dc.robot-learning.net.