这篇论文把RL里训练和测试环境不一致的问题，用因果原因做了系统分类，还统一了OOD泛化和非平稳性的视角，搞清楚偏移根源才能更好做鲁棒性分析。
该论文提出一种基于因果起源的统一分类法，用于描述强化学习（RL）中的分布偏移。作者将监督学习中的经典数据集偏移原则迁移到RL，通过部分可观测马尔可夫决策过程（POMDP）将交互分解为状态分布、观测过程、策略、奖励和转移动力学等结构组件。分类法区分了内部（智能体驱动）和外部（环境驱动）两种分布偏移，并从偏移时间边界角度定义了显式、隐式和混合偏移。该方法统一了分布内/分布外（ID/OOD）泛化与非平稳性，并引入性能退化与恢复指标来评估偏移影响和适应性。
A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learning
Reinforcement learning (RL) systems often degrade when operating conditions differ from those previously encountered, reflecting distributional shifts in the underlying data-generating process. Such shifts may occur between training and evaluation, as in In-Distribution (ID) and Out-of-Distribution (OOD) generalization, or within non-stationary settings where environment dynamics evolve over time. However, the formal relationship between these views remains unclear, and existing work mainly focuses on mitigation rather than the causal origin of shift within the agent-environment interaction. This work develops a unified causal-origin taxonomy that characterizes sources of distributional shift in RL and relates ID/OOD generalization to non-stationary settings. We transfer the classical dataset-shift principle from supervised learning to RL by reformulating distributional shift in terms of the generative interaction process. Using a Partially Observable Markov Decision Process (POMDP), we decompose the interaction into structural components, including the state distribution, observation process, policy, reward, and transition dynamics, together with the shifted-time boundary. The proposed taxonomy distinguishes internal, agent-driven, and external, environment-driven, distributional shifts. The shifted-time boundary perspective further characterizes explicit, implicit, and hybrid shifts. This formulation unifies ID/OOD generalization and non-stationarity as structured changes in the underlying process. We also introduce an evaluation framework for measuring shift impact and adaptation through performance degradation and recovery metrics. By grounding distributional shift in the causal-origin structure of RL, this work supports systematic analysis of robustness under distributional shift.