TreeCCA: 基于梯度提升树的典型相关分析

TreeCCA: Canonical Correlation Analysis via Gradient-Boosted Trees

精选理由

TreeCCA把XGBoost/LightGBM变成CCA编码器,效果不输深度学习,还自带特征重要性解释,适合表格数据多视图分析。

AI 摘要

TreeCCA是首个用梯度提升树端到端训练典型相关分析编码器的方法,通过Eckart-Young损失函数在XGBoost、LightGBM中实现自定义目标。在Signed Power基准上,TreeCCA得分为2.61,超过Deep CCA的2.43;在Hermite上为2.93,接近2.89。在稀疏基准(p=50)上,TreeCCA的Precision@S达到1.00,而PMD无信号。UCI HAR传感器融合任务中,TreeCCA以5倍低成本达到与Deep CCA相当的精度,其增益重要性可直接验证物理假设。

原文 · arXiv cs.LG

TreeCCA: Canonical Correlation Analysis via Gradient-Boosted Trees

Gradient-boosted trees dominate tabular machine learning, yet canonical correlation analysis has always relied on linear or neural encoders. We propose \textbf{TreeCCA}, the first method to train gradient-boosted tree ensembles end-to-end as CCA encoders, inheriting their plug-and-play reliability: no architecture design, familiar hyperparameters, and strong performance with defaults. The technical enabler is the Eckart-Young (EY) loss, which supplies closed-form per-sample gradients that slot directly into any standard GBT library (XGBoost, LightGBM) as a custom objective. TreeCCA is the first CCA method to combine nonlinear accuracy with native interpretability: every tree split selects one feature, so gain importances reveal which inputs drive cross-view correlation at no extra cost. We demonstrate these properties on synthetic benchmarks, where TreeCCA matches or exceeds Deep CCA (2.61 vs.\ 2.43 on Signed Power; 2.93 vs.\ 2.89 on Hermite), and on a sparse benchmark with zero linear cross-view covariance, where TreeCCA recovers the true support with $\text{Precision@}S = 1.00$ at $p=50$ while PMD finds no signal. On the UCI HAR sensor-fusion benchmark, TreeCCA achieves comparable accuracy to Deep CCA at $5\times$ lower cost, while XGBoost gain importances directly validate a physics-motivated hypothesis about the data --- an interpretation not readily available with neural encoders. Across five popular tabular multi-view datasets, TreeMCCA consistently matches or exceeds linear CCA in both nonlinear correlation extraction and downstream classification accuracy.