TabNSM:表格数据神经稀疏混合回归模型

TabNSM: Neural Sparse Mixer for Tabular Regression

精选理由

TabNSM提出了一种新颖的表格回归方法,通过稀疏交互和难度感知采样,解决了深度表格回归中的关键挑战。

AI 摘要

TabNSM是一种针对大规模高维表格数据回归问题的新型框架。该模型包含Adaptive Sparse Interaction Module (ASIM)核心组件,以及Multi-Stage Regression Head、GridLoss和RISE三个互补组件。在9个真实世界回归基准测试中,TabNSM展现出强大的预测性能和实际可扩展性。模型特别在高维和异构数据集上表现一致提升。

原文 · arXiv cs.LG

TabNSM: Neural Sparse Mixer for Tabular Regression

Large-scale, high-dimensional tabular regression remains challenging: tree-based models are robust but lack end-to-end representation learning, while deep models enable flexible feature learning but often incur costly interaction modeling and sensitivity to noisy or redundant features. We propose TabNSM, a scalable regression framework that extends our earlier sparse-attention and mixer architectures. At its core, the Adaptive Sparse Interaction Module (ASIM) integrates foreground feature discovery, sparse local interaction encoding, and Feature-Token Mixing, providing near-linear complexity under fixed sparse configurations. For regression, TabNSM introduces three complementary components: a Multi-Stage Regression Head for progressive prediction refinement; GridLoss, an ordinal-aware soft-binning objective that incorporates target structure into representation learning; and RISE (Reweighted Instance Sampling by Error), a difficulty-aware sampling strategy based on loss-quantile bins. Across nine real-world regression benchmarks, TabNSM delivers strong predictive performance and practical scalability, with particularly consistent gains on high-dimensional and heterogeneous datasets. These results demonstrate that selective interaction modeling, structured regression supervision, and difficulty-aware sampling provide an effective and scalable approach to deep tabular regression.