做表格数据标注或低资源学习的团队,LUCoS解决了冷启动下上下文选择的核心难题,无需标签就能显著提升模型效果,值得在TabPFN等模型上试试。
在低标注表格学习场景中,如何选择标注实例是关键挑战。对于TabPFN等表格基础模型,上下文选择直接影响预测性能。有监督实验表明,精心选择的标注集能显著优于随机选择。但冷启动场景(无标签时选择实例)研究不足。LUCoS方法利用无监督预训练网络的潜在几何结构选择代表性样本作为上下文,在67个数据集上平均AUC、ACC和F1排名第一。该方法通过覆盖度和表示空间的选择机制,有效避免了原始特征空间选择失效的问题。
LUCoS: Latent Unsupervised Context Selection for Tabular Foundation Models
Selecting which instances to label is a key challenge in low-label tabular learning. For recent Tabular Foundation Models such as TabPFN, context selection directly determines predictive performance. Supervised oracle experiments show that carefully chosen labeled context sets can strongly outperform random selection under the same labeling budget. However, the cold-start setting, where instances must be selected before any labels are available, has received little attention in the TFM literature. This problem is fundamentally geometric. In vision and language, foundation models induce embedding spaces where simple geometric selection methods are effective. In contrast, tabular instance selection has so far been performed predominantly in the original tabular space, which lacks a natural metric; heterogeneous types, mixed scales, and nonlinear interactions make raw-space distances unreliable for context construction, and original-space selection falls below random on the majority of datasets as the budget grows. We propose LUCoS (Latent Unsupervised Context Selection), which replaces raw-feature geometry with the latent geometry induced by embeddings from an unsupervised Prior-Fitted Network (PFN) and selects representative medoids as context. Evaluated on 67 OpenML-CC18 datasets across six low-label budgets, LUCoS ranks first under mean AUC, ACC, and F1, with conclusions stable across metrics and dataset-level robustness checks. A gain decomposition reveals a simple mechanism: at the smallest budgets, the main benefit comes from enforcing coverage; as the budget increases, the decisive factor becomes the representation space in which coverage is measured. LUCoS mitigates failures of original feature space selection, showing that reliable unsupervised context selection depends less on selector sophistication than on defining representativeness in a meaningful representation geometry.