多模态提示学习框架用 3D CT 预测肺腺癌浸润性
Multimodal Prompt Learning with Dual-Granularity Contrastive Alignment for Preoperative Prediction of Lung Adenocarcinoma Invasiveness Using 3D CT
医学影像方向的朋友可以看看：用报告和关键词做双粒度对齐，1,593 个结节上 AUC 从 0.9297 拉到 0.9499，推理还只要 CT。
研究提出一个多模态提示学习框架，用 3D CT 区分浸润性肺腺癌（IAD）与浸润前病变。框架采用 3D ResNet-CBAM 编码器提取视觉特征，冻结的 RadBERT 骨干通过报告级和关键词级两组可学习提示做文本编码，并结合全局对比对齐与基于多示例学习的局部对齐。在 1,593 个病理确认结节的测试集上，准确率 87.92%，F1 87.78%，AUC 0.9499，显著高于同视觉骨干的纯图像基线（AUC 0.9297，DeLong 检验 p=0.0078）。推理阶段只需 CT 图像。
Multimodal Prompt Learning with Dual-Granularity Contrastive Alignment for Preoperative Prediction of Lung Adenocarcinoma Invasiveness Using 3D CT
Accurate preoperative prediction of lung adenocarcinoma invasiveness is important for surgical planning and individualized treatment. However, overlapping CT appearances make it challenging to distinguish invasive adenocarcinoma (IAD) from Pre-IAD lesions. We propose a multimodal prompt-learning framework with dual-granularity image-text alignment for 3D CT-based IAD classification. A 3D ResNet-CBAM encoder extracts global and spatial visual features. A frozen RadBERT backbone is adapted with separate learnable prompts for report-level and keyword-level text encoding. These prompts are initialized using semantic representations derived from real-world clinical reports. Global contrastive alignment connects whole-volume CT representations with reports generated from clinician-annotated radiological attributes to provide overall semantic supervision. A multiple instance learning-based local alignment module associates spatial CT features with vascular-focused radiological keywords, to capture local imaging characteristics potentially associated with tumor invasion. The framework jointly optimizes classification and alignment objectives, while inference requires only CT images. Experiments included 1,593 pathologically confirmed nodules. On the test set, the proposed method achieved an accuracy of 87.92%, F1-score of 87.78%, and AUC of 0.9499. Its AUC significantly exceeded that of the image-only Res-CBAM baseline with the same visual backbone (0.9297; DeLong test, p = 0.0078). Ablation analyses supported the complementary contributions of global and local alignment and the benefit of report-based prompt initialization. Qualitative visualizations showed activation within nodules and adjacent vascular structures. These findings suggest that dual-granularity radiological supervision can enhance CT-based invasiveness prediction while preserving a CT-only inference workflow.