因果 PFN 新训练方法:联合学习事实与反事实世界
Joint Learning of Factual and Counterfactual Worlds for Causal Prior-data Fitted Networks
这篇论文解决的是因果 PFN 训练里的一个结构缺陷:明明是纯合成数据,却只学单一世界的结果。作者把事实与反事实联合建模,再用最优传输算平均处理效应。做因果推断或评估 ATE 的研究者可以直接借鉴。
一篇 arXiv 论文(编号 2611.19710)提出 joint learning 训练方法,让因果先验数据拟合网络(causal PFN)同时学习每个单元在事实世界与反事实世界的联合结果分布。此前的 causal PFN 虽然完全在合成数据上预训练,却把真实数据中每个单元只能观测单一世界结果的限制带进了训练目标,导致两个世界之间的依赖关系丢失,处理效应密度只能默认假设独立性。
新方法从联合分布中同时导出响应密度、个体处理效应密度,并在推理阶段通过跨单元的最优传输近似平均处理效应密度,是首个同时提供三种密度并考虑跨世界依赖的 causal PFN 方法。在半合成、合成与真实基准上评估,joint learning 训练的模型在点估计和不确定性量化上多数显著优于基线。
Joint Learning of Factual and Counterfactual Worlds for Causal Prior-data Fitted Networks
Estimating treatment effects is central to science across many disciplines and requires counterfactual reasoning about alternative worlds. In the real world, however, each unit reveals its outcome only under the factual world. Thus, the factual and counterfactual outcomes can never be jointly learned by any model trained on real data. The problem is therefore tractable only in synthetic settings, where both worlds can be generated. Recent progress on prior-data fitted networks (PFNs) has shown that models pre-trained purely on synthetic data can transfer to real-world prediction. This paradigm has since carried over to causal effect estimation. Despite this fully synthetic setup, existing causal PFNs carry the real-world constraint into their training objective, confining learning to a single world's outcome per unit. The dependence between the two worlds is thereby lost, and the density of the treatment effect relies by default on independence, which is implausible in practice. We therefore propose joint learning, a training setup in which a causal PFN learns the joint distribution of the factual and counterfactual outcomes of each unit. From the joint distribution, we obtain both the response and the individual treatment effect densities, and at inference, we apply optimal transport across units to obtain an approximation of the average treatment effect density. Ours is therefore the first causal PFN approach to provide all three densities together while accounting for the cross-world dependence. We apply joint learning and optimal transport to existing causal PFNs and evaluate it across semi-synthetic, synthetic, and real benchmarks spanning diverse causal scenarios. The jointly trained models significantly improve upon their baselines in point estimation and uncertainty quantification in most settings, indicating that joint learning is a preferable training setup for causal PFNs.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include PFN, causal inference, optimal transport, papers.
Source attribution: curated from arXiv cs.AI (@Furkan Danisman, Luke Zhang, Zhijing Jin, Dehan Kong).