新论文用语义感知对比学习解决医学影像的假阴性问题，在脑肿瘤分类上AUC涨了22.6%，值得搞多模态医疗AI的人看看。
该论文提出MseaCL框架，在儿科3D脑部MRI和放射学报告的多模态对比学习中，通过语义相似性引导减少假阴性样本。传统对比学习将非配对样本视为负例，但医学影像中语义相似的样本常被误判为负例。MseaCL在儿科队列上训练，将报告语义相似度作为信号，下游任务中儿科脑肿瘤分子分类AUC提升至少22.6%。
Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging
Multimodal Contrastive Learning (CL) has shown significant performance in aligning representations across various data modalities and improving downstream tasks, especially in healthcare. It works by minimizing the distance between matched (positive) data modalities, while maximizing the distance between mismatched (negative) samples. Traditional CL frameworks typically assume instance-based correspondence within data batches, treating all non-paired samples as negatives. However, this assumption often fails in medical settings, where samples may share high-level semantic attributes, leading to false negatives that degrade representation quality. In this paper, we propose Multimodal Semantic-Aware Contrastive Learning (MseaCL), a CL framework trained on a pediatric cohort of 3D brain magnetic resonance imaging (MRI) scans and radiology reports. The goal of this framework is to mitigate the impact of semantically similar false negative samples by incorporating semantic similarity between radiology reports, as a guiding signal during the learning process. Our results indicate that applying this framework as a pretraining stage can achieve notable improvements in downstream tasks, e.g., at least a 22.6\% increase in the area under the receiver operating characteristic curve (AUC) of pediatric brain tumor molecular classification, demonstrating its potential for more robust and semantically aligned multimodal representations in clinical applications.