想用LoRA做多模态融合？这篇用级联方式逐步整合不同模态，在医疗动作识别上效果不错，还开源了代码，可以看看。
该论文提出一种基于低秩适配（LoRA）的级联多模态融合框架，用于医疗培训环境中的动作识别。框架先整合相关性高的模态（如RGB、骨骼），再逐步加入异质模态（如音频），无需重新训练先前组件。在NurViD和Nurse Training两个数据集上评估，级联融合策略优于单模态模型，与先前报告的数据集特定基线（如基于Transformer的方法）性能相当。代码已在GitHub开源。
LoRA-Based Cascaded Multimodal Fusion for Action Recognition in Medical Training Environments
This paper presents a cascaded Low-Rank Adaptation (LoRA)-based multimodal fusion framework for action and activity recognition in healthcare-oriented training environments. The proposed architecture combines parameter-efficient modality-specific adaptation with sequential fusion, enabling modalities to be integrated in stages without retraining previously learned components. Rather than assuming a fixed fusion structure, the framework first integrates more closely related modalities and then incorporates additional heterogeneous modalities, supporting scalable adaptation across datasets with different modality sets.We evaluate the framework on two healthcare-oriented training environment datasets: NurViD and the Nurse Training dataset. Across these datasets, preliminary results suggest that the proposed cascaded fusion strategy improves over individual modality models and provides competitive performance relative to previously reported dataset-specific baselines. Overall, these findings indicate that cascaded LoRA-based fusion is a promising parameter-efficient approach for integrating heterogeneous modalities in medical training action and activity recognition tasks. github: https://github.com/anonymous0-ai/LoRA-Based-Cascaded-Multimodal-Fusion-.git.