这篇论文指出了现有offline RL治疗推荐研究可能不靠谱，因为协变量平衡诊断不够用，搞因果推断的人可以看看。
该论文探究了协变量平衡诊断在离线强化学习（offline RL）应用于最优治疗推荐时检测隐藏混淆/模型误设的有效性。结果显示，现有offline RL治疗推荐研究要么存在高偏倚风险，要么现有协变量平衡指标不足以评估这类研究。因此，现有offline RL研究无法被确认为统计上稳健。论文提出了未来研究方向以提升offline RL在治疗推荐问题中的方法学稳健性。
Evaluating covariate balance for long time horizon Markov decision processes
This article explores the application of covariate balance diagnostics for detecting the presence of hidden confounding/model miss-specification in studies applying offline reinforcement learning (RL) to deriving optimal treatment recommendations. The results demonstrate that, either there is a high risk of bias within existing offline RL studies for treatment recommendations or, existing covariate balance metrics are not sufficient to assess such studies. Regardless, existing offline RL studies cannot be concluded as being statistically robust. The conclusions propose future research directions for obtaining more methodologically robust applications of offline RL to treatment recommendation problems.