TCLA:无需训练的类别级逻辑适配方法用于医学视觉-语言模型

TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models

精选理由

这篇论文提出了TCLA方法,不用训练就能让医学VLMs适应新数据,1-shot下效果比现有训练方法还好,很实用。

AI 摘要

TCLA是一种纯训练自由的少样本适应方法,针对医学视觉-语言模型(VLMs)在分布外(OOD)数据上性能下降的问题。该方法仅需少量支持样本即可校正推理逻辑,无需额外训练组件,在1-shot场景下依然稳定。在9个数据集(涵盖X光、超声、MRI、CT、组织病理学等模态)上,TCLA持续提升OOD性能,多数情况下超越现有基于训练的方法。

原文 · arXiv cs.AI

TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models

Medical Vision-Language Models (VLMs) exhibit strong zero-shot performance, yet their effectiveness still declines on out-of-distribution (OOD) data due to domain shifts and class bias inherited from large-scale pretraining. Existing few-shot adaptation methods typically introduce additional trainable components, which can be unstable in extremely low-data regimes (e.g., 1-shot), and lack robustness on different medical data. We present TCLA, a purely training-free few-shot adaptation method for Medical VLMs, which is fast and model-agnostic. TCLA corrects inference logits based on a small set of support samples, boosting pretrained VLMs performance by improving inter-class deconfusion and reducing domain shift. Extensive experiments on nine datasets across multiple medical imaging modalities including X-ray, Ultrasound, MRI, CT, Histopathology, demonstrate that TCLA consistently improves OOD performance of Medical VLMs and, in most of cases, outperforms existing training-based adaptation methods.