这篇论文提出了一个叫FSDIL的新问题和一个叫CVLC的算法，用双融合投影微调视觉语言模型，在少样本领域增量学习上比之前的方法好16%。
本文定义了少样本领域增量学习（FSDIL）问题，针对传统方法在数据极少时过拟合的痛点。提出持续视觉-语言整合（CVLC）算法，关键思想是在基领域预留隐空间和双融合投影（DCP）参数高效微调。通过LLM生成多模板和同义词校准视觉与语言原型并融合，再用DCP适应新领域。在多个基准上，CVLC相比此前方法提升最高达16%。代码已开源。
Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidation
Existing domain-incremental learning (DIL) strategies call for massive amounts of data to adapt to new domains and suffer from the overfitting problem in the case of data scarcity. This paper puts forward a relatively uncharted problem, namely, few-shot domain incremental learning (FSDIL), taking into account the problem of extreme data shortages in the realm of DIL. A novel algorithm, namely Continual Vision-Language Consolidation (CVLC), is proposed to address the FSDIL problem, where the key idea lies in the concept of latent space reservation in the base domain coupled with dual coalescent projection (DCP) as a parameter-efficient fine-tuning method. First, the vision prototype is calibrated while multiple templates and synonyms are generated via LLMs to induce the language prototype. The vision and language prototypes are fused. Adaptation to never-ending arrivals of new domains is done by the DCP technique, fine-tuned in such a way to prepare the model to unseen domains via latent-space reservations committed in the base domain. CVLC is structured under shared and domain-specific components to combine general knowledge and domain-specific details. The advantage of our approach is demonstrated through a range of benchmark problems and comparisons with prior arts, in which CVLC outperforms them by up to a 16% gap. Our codes are shared publicly in https://github.com/Naeem-Paeedeh/CVLC .