这篇论文提出了在MORL中兼顾最优和公平的新方法，用GGF和三种算法生成多样化的公平策略，比传统单策略方法更灵活。
该论文研究多目标强化学习（MORL）中的公平性优化问题。传统单策略方法使用广义基尼福利函数（GGF）只能处理固定用户偏好，缺乏策略多样性。作者证明对于凹分段线性福利函数（如GGF），公平策略仍属于凸覆盖集（CCS）。提出三种新算法：集成GGF的多策略多目标Q学习（MOQL）、状态增强多策略MOQL及随机策略扩展。在多个领域实验表明，该方法能学习一组适应不同用户偏好的公平帕累托最优策略。
Learning Fair Pareto-Optimal Policies in Multi-Objective Reinforcement Learning
Fairness is an important aspect of decision-making in multi-objective reinforcement learning (MORL), where policies must ensure both optimality and equity across multiple, potentially conflicting objectives. While single-policy MORL methods can learn fair policies for fixed user preferences using welfare functions such as the generalized Gini welfare function (GGF), they fail to provide the diverse set of policies necessary for dynamic or unknown user preferences. To address this limitation, we formalize the fair optimization problem in multi-policy MORL, where the goal is to learn a set of Pareto-optimal policies that ensure fairness across all possible user preferences. Our key technical contributions are threefold: (1) We show that for concave, piecewise-linear welfare functions (e.g., GGF), fair policies remain in the convex coverage set (CCS), which is an approximated Pareto front for linear scalarization. (2) We demonstrate that non-stationary policies, augmented with accrued reward histories, and stochastic policies improve fairness by dynamically adapting to historical inequities. (3) We propose three novel algorithms, which include integrating GGF with multi-policy multi-objective Q-Learning (MOQL), state-augmented multi-policy MOQL for learning non-statoinary policies, and its novel extension for learning stochastic policies. We evaluate our algorithms across various domains and compare our methods against the state-of-the-art MORL baselines. The empirical results show that our methods learn a set of fair policies that accommodate different user preferences.