医疗AI团队终于有了低成本部署高精度表格模型的方案——蒸馏后模型保留90%性能且快26倍，做健康数据预测的开发者可以直接用。
表格基础模型在健康数据集上表现优异，但高推理成本和基础设施需求限制了实际应用。研究者提出通过知识蒸馏将预测能力转移至轻量表格模型，并针对上下文表格模型在推理时依赖训练集导致的上下文泄露问题，采用分层折叠教师标注策略。在19个医疗数据集、6个教师模型、4个学生模型家族及多教师集成实验中，蒸馏学生模型保留了教师AUC的至少90%，部分甚至超越教师，同时CPU推理速度提升至少26倍，且保持校准性和公平性。多教师平均并未持续优于最佳单教师。该研究为推理受限的健康场景部署高质量预测提供了可行路径。
Distilling Tabular Foundation Models for Structured Health Data
Tabular foundation models (TFMs) achieve strong performance on health datasets, but their inference cost and infrastructure requirements limit practical use. We study whether their predictive behavior can be transferred to lightweight tabular models through knowledge distillation. Since in-context TFMs condition on the training set at inference time, naive distillation can introduce context leakage; we address this with stratified out-of-fold teacher labeling. Across $19$ healthcare datasets, $6$ TFM teachers, $4$ student families, and several multi-teacher ensembles, we find that distilled students retain at least $90\%$ of teacher AUC, outperforming teachers in some cases, while running at least $26\times$ faster on CPU and preserving calibration and fairness critical for health applications. Moreover, multi-teacher averaging does not consistently improve over the best single teacher. Leakage-aware distillation is thus a viable route for bringing TFM-quality predictions into inference-constrained health settings.