临床预测中 VLM 对 ECG 的利用不足问题及缓解方法研究
ECG Mirage: Revealing and Mitigating the Underutilisation of ECGs in Vision-Language Models for Clinical Prediction
朋友，这是篇挺有意思的论文，研究的是临床预测中 VLM 模型如何利用 ECG 数据。他们发现模型虽然能处理多模态信息，但可能没真正用到 ECG，然后提出了一种叫视觉提示词调优的方法来解决这个问题。
研究指出，在临床预测任务中，四款 VLM 模型（MDS-ED 数据集）使用匹配 ECG 输入时，对 ICU 入院或病情恶化的预测准确率提升并不显著。研究通过训练受限视觉提示词，将匹配与不匹配 ECG 的预测性能差距扩大至约 16.5 和 5.5 个百分点，以此缓解 ECG Mirage 问题。
ECG Mirage: Revealing and Mitigating the Underutilisation of ECGs in Vision-Language Models for Clinical Prediction
Emergency department (ED) decision-making relies on heterogeneous clinical information, including patient history, vital signs, laboratory results, and electrocardiograms (ECGs). Vision--language models (VLMs) can jointly process these modalities, but strong predictive performance does not necessarily imply meaningful use of the correct patient's ECG. We term this failure mode ECG Mirage: apparent multimodal capability without useful dependence on patient-specific ECG information. We distinguish two forms: ECG neglect, where ECGs provide little predictive benefit, and ECG confusion, where matched ECGs outperform no-image inputs but not mismatched ECGs. To evaluate these behaviours, we compare predictions obtained with matched ECGs, outcome-discordant mismatched ECGs, and no-image inputs while holding the clinical text and prediction targets fixed. Across four VLMs on MDS-ED, matched ECGs provide no consistent advantage for either ICU admission or clinical deterioration prediction. We then train four restricted visual prompts using supervised learning followed by conditional direct preference optimisation, while keeping the VLM backbone frozen. The resulting models achieve balanced accuracies of 70.6% for ICU admission and 67.5% for deterioration and increase the matched-versus-mismatched performance gap to approximately 16.5 and 5.5 percentage points, respectively. Overall, our study identifies ECG Mirage in multimodal clinical prediction and introduces visual prompt tuning as an efficient mitigation strategy.