这篇paper教你如何用A/B测试的思路检测预测本身会不会改变结果，有理论有实验，做因果推断或者推荐系统的值得一看。
论文提出结果表现性A/B检测（OPAB）方法，通过比较不同预测组（干预）产生的结果分布差异来检测结果表现性。在姑息治疗、信用分配和推荐系统等场景中，预测会因果影响其预测结果，即结果表现性。研究推导了OPAB在多种假设类别下的样本复杂度边界，并通过实验验证。实验显示OPAB在多数情况下可检测结果表现性，但也存在不可区分区域，即干预数量不足时无法检测。在Open Bandits数据集上的案例研究验证了OPAB的有效性。
Actions Have Consequences: Detecting Outcome Performativity using Intervention Testing
In many domains such as Palliative Care, Credit Assignment and Recommender Systems, predictions may causally influence the outcomes they predict. This phenomena is known as Outcome Performativity. This paper formalises an approach for detecting Outcome Performativity using prediction intervention called Outcome Performativity A/B Detection (OPAB). OPAB enables the detection of Outcome Performativity by assessing the dissimilarity in outcome distributions produced by different predictions groups (interventions). If that dissimilarity is significant, Outcome Performativity is detected. We derive sample complexity bounds for OPAB under various Outcome Performative assumption classes which we empirically validate. Results show that detecting Outcome Performativity using OPAB is achievable in numerous cases. Results also show the presence of regions of indistinguishability which describe settings where the allotted number of interventions are insufficient for detecting Outcome Performativity. The results of which have broader practical implications for the detectability of Outcome Performativity in settings where samples are scarce, cost-prohibitive or potentially unethical to obtain. The paper concludes with a case study on the efficacy of OPAB on the Open Bandits dataset, and provides directions for future work.