Prism解决了MCIT研究中代码碎片化和复现难的问题,做多模态持续学习的团队可以直接用这个插件式框架加速实验,省去大量工程重复劳动。
多模态大语言模型(MLLMs)通过指令调优将多样任务统一为指令遵循框架,但实际部署需要持续适应新任务,这催生了多模态持续指令调优(MCIT)。然而,当前MCIT研究受限于工程瓶颈:现有方法通常直接修改基础MLLM代码库,导致实现开销大、架构特定、代码复用和公平比较困难。为此,研究者提出Prism,一个插件式可复现代码库,通过轻量级插件注册机制将算法开发与骨干实现分离,无需修改底层MLLM代码即可集成新策略。Prism原生支持大规模训练流水线,确保可复现和可扩展的MCIT实验。代码已开源。
Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning
Multimodal Large Language Models (MLLMs) achieve versatility by reformulating diverse tasks into a unified instruction-following framework via instruction tuning. However, real-world deployment requires continuous adaptation to emerging tasks, motivating Multimodal Continual Instruction Tuning (MCIT). Despite its growing importance, current MCIT research is hindered by severe engineering bottlenecks. Existing methods are typically implemented by directly modifying the base MLLM codebase, which imposes substantial implementation overhead and yields method-specific architectures that severely limit code reuse and fair comparison. To address this, we introduce Prism, a plug-in reproducible codebase specifically designed for scalable MCIT research. It separates algorithmic development from the backbone implementation via a lightweight plugin registration mechanism, enabling new strategies to be integrated as independent plugins without modifying the underlying MLLM codebase, thereby eliminating structural fragmentation and accelerating method development. Prism natively supports widely used large-scale training pipeline, thereby enabling reproducible and scalable MCIT experimentation. Code is available at https://github.com/LAMDA-CL/Prism.