Papers

Multi-Objective Causal Bandits:策略层面学习与最小干预空间新界定

Multi-Objective Causal Bandits: Minimal Intervention Space and Policy-Level Learning

Why we picked it

搞多目标决策或 bandit 研究的可以看看。它证明了被随机策略占优这个此前被忽略的现象,还把干预空间剪枝做到了图上的充要刻画,配套的 UCB 算法有对数遗憾证明,做公平性或多约束场景的会用得上。

一篇 arXiv 论文研究了带多个奖励节点的多目标因果 bandit(multi-objective causal bandits),动作对应因果图上变量子集的干预,性能用 Pareto 最优性评估。

Body

作者指出仅靠成对 Pareto 占优比较不够:一个干预可能不被任何单个干预占优,却会被其他干预的随机化策略(凸组合)占优。论文给出可能 Pareto 最优最小干预集(minimal intervention sets)的充要图刻画,纠正了先前会纳入永不 Pareto 最优干预集的公式化,从图结构即可得到最小且可靠的候选干预族。

在此基础上,作者设计了 UCB 风格的在线算法,用与其他动作凸组合的占优测试淘汰动作,证明了依赖间隔的对数级 Pareto 遗憾;还针对随机化策略才能同时实现可行与最优的约束场景,提出带次线性遗憾与约束违反保证的约束 bandit 算法。

Original · arXiv cs.LG

Multi-Objective Causal Bandits: Minimal Intervention Space and Policy-Level Learning

Many decision-making problems involve multiple objectives, where actions affect several performance metrics, fairness criteria, or safety constraints. We study multi-objective causal bandits with multiple reward nodes, where actions correspond to interventions on subsets of variables in a causal graph and performance is evaluated through Pareto optimality. Existing approaches to multi-objective bandits often rely on pairwise Pareto dominance or optimize a fixed scalarization. However, pairwise comparisons alone can fail to identify actions dominated by randomized policies, whose expected reward vectors lie in the convex hull of action-level rewards. Thus, an intervention may be suboptimal even if no single intervention dominates it, because it can be dominated by a randomized policy over other interventions. We formalize this policy-level phenomenon and show that it has consequences for both causal action-space reduction and online learning. First, we provide a necessary-and-sufficient graphical characterization of possibly Pareto- optimal minimal intervention sets. Our characterization yields a minimal, sound, and complete candidate intervention family from the graph alone, correcting prior formulations that may include intervention sets that are never Pareto-optimal under any compatible structural causal model. Second, over this reduced intervention space, we design an online UCB-style algorithm that eliminates actions using dominance tests against convex combinations of other actions, and prove logarithmic gap-dependent Pareto regret. Finally, we study constrained multi-objective causal bandits, where feasibility and optimality may be realized only with randomized policies, and develop an online UCB-style constrained bandit algorithm with sublinear regret and constraint-violation guarantees.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include causal inference, multi-objective optimization, bandit, reinforcement learning.

Source attribution: curated from arXiv cs.LG (@Muhammad Qasim Elahi, Mahsa Ghasemi, Murat Kocaoglu).