ATLAS:从异构环境中发现不变和可迁移潜在因子

Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAS

精选理由

想从多环境数据中提取通用因子做迁移学习?这篇论文提出的ATLAS方法能分离不变和可迁移因子,适合做跨环境预测的研究者。

AI 摘要

论文提出多环境因子模型,高维协变量来自异构环境,部分环境有辅助标签。模型将潜在结构分解为不变因子(共享载荷)和异构因子(环境特定载荷)。基于不变性原理,ATLAS方法分离对齐的不变因子和未对齐的异构因子,并利用辅助标签从异构因子中提取预测不变和可迁移因子。ATLAS在下游潜在因子回归中达到近最优性能,在辅助标签可用时实现全潜在信号的可迁移预测,否则退化为鲁棒的不变因子预测。论文建立了恢复不变和异构因子的尖锐非渐近误差界。

原文 · arXiv cs.LG

Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAS

This paper considers a multi-environment factor model in which high-dimensional covariates are collected from heterogeneous environments, with auxiliary labels available in a subset of these environments. The joint distribution of the covariates may vary across environments, whereas the latent structure is decomposed into invariant factors with shared loadings and heterogeneous factors with environment-specific loadings. Such a model is motivated by transfer learning and latent factor regression, where one seeks stable low-dimensional representations for both interpretation and robust out-of-sample prediction of the response $Y$. Leveraging the invariance principle, we show that the invariant and heterogeneous factors are disentangled under a minimal structural condition. Based on this, we propose ATLAS, an Auxiliary-label and invariance-guided Transfer via Latent Alignment across heterogeneous environmentS. ATLAS is a unified procedure that leverages the invariance principle to separate aligned invariant and unaligned heterogeneous factors, and further exploits supervision from auxiliary labels to extract prediction-invariant and transferable factors from those unaligned heterogeneous factors. ATLAS yields near-oracle performance for downstream latent factor regression, enables transferable prediction in new environments through the full latent signal when auxiliary labels are available, and reduces to robust invariant-factor-only prediction otherwise. We establish sharp non-asymptotic error bounds for recovering invariant and heterogeneous factors, identifying all the response-invariant factors, and estimating the invariant signal in $Y$.