处理表格数据的团队终于有了区分“不该填”和“该填”缺失值的工具——Diff-Joint解决了传统填补方法盲目恢复所有缺失的痛点,做数据清洗或医疗、金融等缺失值有语义含义的开发者可以直接试。
传统缺失值填补方法假设所有缺失都是随机且应被恢复,但现实中缺失可能来自两种不同来源:有意义缺失(数据本身不存在)和观测缺失(应被填补)。研究者提出Diff-Joint,一个基于扩散的框架,联合建模表格数据与潜在缺失掩码,通过条件采样和不确定性感知聚合迭代优化填补值与缺失标签。实验表明,该方法能有效识别有意义缺失,同时保持竞争性填补精度并提升下游任务性能。
Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness
Missing value imputation is a fundamental task in machine learning, with most existing methods assuming that all missing entries correspond to unobserved regular values. In many real-world datasets, however, missingness may arise from two distinct sources: some entries are meaningfully missing (intrinsically absent and semantically valid), while others are missing due to the observation process and should be imputed. We formalize this distinction as a selective imputation problem, where the goal is to jointly infer which missing entries should be preserved and which should be recovered. To address this challenge, we propose Diff-Joint, a diffusion-based framework that jointly models tabular data together with a latent missingness mask. The method alternates between conditional sampling and uncertainty-aware aggregation to iteratively refine both imputed values and missingness labels. Empirical results on synthetic and real-world datasets demonstrate that Diff-Joint effectively identifies meaningfully missing entries while achieving competitive imputation accuracy and improved downstream task performance.