安全多目标强化学习：全程零约束违反与收敛保证
Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence
搞强化学习又怕策略踩安全红线的看这篇，零约束违反加收敛保证都给了。
论文研究安全多目标强化学习（MORL），要求策略在学习全程满足累计与分步两类安全约束。算法将乐观奖励估计与悲观成本界限结合，对给定偏好学习安全的 Pareto 最优策略，在 T 个回合内以高概率保证每条执行策略零约束违反，并在平滑 Tchebycheff 标量化下取得 O(1/√T) 次优差距。另一版无偏好算法先安全探索再离线规划，在 T 个探索回合与 K 次规划迭代下达到 O(1/√T+1/√K) 次优差距。作者称这是首批对每条执行策略都提供安全保证的样本高效 MORL 算法。
Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence
Many applications of multi-objective reinforcement learning (MORL) require policies that balance conflicting objectives while satisfying safety constraints. We study safe MORL with zero constraint violation throughout learning, covering both cumulative and stepwise constraints. A key challenge is to explore sufficiently under unknown dynamics and constraint costs without violating the constraints. For a given preference over objectives, we develop an algorithm to learn a safe Pareto optimal policy by combining optimistic reward estimates with pessimistic cost bounds. With high probability, every policy executed over $T$ episodes incurs zero constraint violation. The algorithm returns a safe Markov policy induced by the averaged occupancy measure and achieves an $\widetilde{O}(1/\sqrt{T})$ suboptimality gap under smooth Tchebycheff scalarization. We further develop a preference-free algorithm that first explores safely without a prespecified preference and then computes a safe policy for any given preference without further interaction. Over $T$ exploration episodes and $K$ planning iterations, it retains the same safety guarantee and achieves an $\widetilde{O}(1/\sqrt{T}+1/\sqrt{K})$ suboptimality gap. Both algorithms require knowledge of a policy that strictly satisfies the constraints and its costs. We then relax these requirements, incurring at most constant cumulative violation. To our knowledge, we provide the first sample-efficient MORL algorithms that guarantee safety for every executed policy.