Tydra:高效混合模型用于表格数据

Tydra: An Efficient Hybrid Model for Tabular Data

精选理由

Tydra模型在保持预测性能的同时大幅降低推理时间,比Hydra模型更高效。

AI 摘要

Tydra,一种混合Transformer-State Space Model架构,在30个OpenML数据集上,相较于TabPFN减少30%的推理时间,同时保持大部分预测性能,优于Hydra模型,表明混合架构是表格基础模型的有前景方向。

原文 · arXiv cs.LG

Tydra: An Efficient Hybrid Model for Tabular Data

Transformer-based tabular foundation models such as TabPFN achieve strong predictive performance but incur quadratic computational cost with context length. On the other hand, subquadratic SSM-based alternatives such as Hydra trade away accuracy for efficiency. To balance both, we introduce Tydra, a hybrid Transformer-State Space Model (SSM) architecture for tabular in-context learning that interleaves attention and SSM layers. Across 30 OpenML datasets, Tydra reduces inference time by 30% relative to TabPFN while retaining much of its predictive performance. Tydra also outperforms an approximately ten-times-larger Hydra model while providing faster inference. The results indicate that hybrid architectures are a promising direction for tabular foundation models.