想知道不同表格基础模型的数据先验到底差多少?这篇论文用统一框架帮你测了一遍,结果挺有意思。
该研究提出一个统一接口,用于比较不同表格基础模型的数据先验。他们从TabPFN、TabR等公开先验和真实数据集生成训练任务,在固定架构和训练协议下训练模型。在共享下游分类任务上评估发现不同先验偏好不同任务特性,部分先验绝对性能更强,部分排序更稳定。数据级相似度仅能部分解释下游行为差异。
Towards Evaluating Data Priors for Tabular Foundation Models
Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However, priors are rarely evaluated as independent components, making it difficult to understand how much they affect downstream model behavior. This raises a methodological question: how can priors from different tabular foundation models be compared independently of the architectures and training protocols they were introduced with? To study this question, we implement a unified interface for publicly available priors from recent tabular foundation models and priors constructed from real datasets. We generate training tasks from each prior, train the same model architecture under a fixed training protocol, and evaluate the resulting models on shared downstream classification tasks. We compare priors through both generated-task statistics and downstream predictive performance. Our results show that different priors favor different downstream behaviors, with some achieving stronger absolute performance and others exhibiting more consistent relative rankings across datasets. We further find that data-level similarity only partially explains downstream behavior. Our code is available at https://github.com/automl/TFM-Playground/tree/prior-dev.