这篇论文系统比较了随机森林的各种分布分裂方法,发现各向同性MMD就够了,多变量响应才是分布分裂的用武之地,干货多结论扎实。
该研究在诚实森林框架内系统实现并比较了多种分布分裂准则,包括各向同性随机傅里叶特征最大平均差异(MMD)、各向异性对角带宽变体、自适应逐分裂频率选择变体以及切片Wasserstein准则,并测试了后验核均值收缩。通过配对种子比较,在合成分位数机制、真实单变量基准、加州住房子样本曲线以及多变量合成和真实响应上评估各扩展的性能。三个主要发现:各向同性MMD已接近最优,基于均值的CART分裂在标量表格回归中仍是稳健默认,多变量响应是分布分裂的明确优势场景,其中能量得分能区分各准则而边际CRPS不能。所有准则和森林实现已开源在drforest库中。
Distributional Split Criteria for Random Forests: Extensions, Shrinkage, and the Robustness of Mean Splitting
Distributional random forests replace mean-based CART splitting with criteria that compare the full conditional response distribution in candidate children. We implement and systematically study a family of such criteria inside a single honest-forest implementation: isotropic random-Fourier-feature maximum mean discrepancy (MMD), an anisotropic diagonal-bandwidth variant, an adaptive per-split frequency-selection variant, and a non-kernel sliced-Wasserstein criterion, together with post-hoc kernel-mean shrinkage of the forest weights. Using paired-seed comparisons across synthetic quantile mechanisms, real univariate benchmarks, a California-housing subsample curve, and multivariate synthetic and real responses, we characterize where each extension pays. Three findings recur. First, among distributional criteria ordinary isotropic MMD is already close to best in class: the anisotropic, adaptive-frequency, and sliced-Wasserstein extensions, and post-hoc shrinkage, do not systematically improve on it. Second, on scalar tabular regression mean-based CART splitting remains the robust default and wins many cells. Third, multivariate responses are the regime where distributional splitting clearly earns its keep, most sharply on a pure-dependence copula where the energy score separates the criteria even though marginal CRPS does not. The evidence supports a simple allocation story: distributional splitting helps only when non-location structure is both present and estimable; otherwise it dilutes split-selection power away from the mean. All criteria, the honest forest, and the paired-comparison harness are implemented in the open-source \texttt{drforest} library, whose Rust-backed split search makes broad criterion sweeps inexpensive.