Papers

ERIM 框架让多模态上下文学习显式归纳规则

Multimodal In-Context Learning with Explicit Rule Induction

Why we picked it

一篇教多模态模型先从示例里总结规则再答题的论文,相比直接映射输入输出,性能更稳、可解释性也更强。

论文提出 ERIM(Explicit Rule-Induction for Multimodal In-Context Learning),针对现有多模态上下文学习方法只做输入输出映射、不归纳规则的问题。ERIM 借助多模态大语言模型(MLLM)构建以规则为中心的数据集,通过冷启动监督微调加强化学习的两阶段训练实现显式规则归纳监督。实验显示该方法在多种 MICL 设置下带来一致的性能提升,正确归纳规则能换来更强且更稳定的表现。

Original · arXiv cs.AI

Multimodal In-Context Learning with Explicit Rule Induction

A central challenge in multimodal in-context learning (MICL) arises from tasks that are ill-defined without context and require uncovering the underlying rules from demonstrations. Existing methods predominantly treat MICL as direct input-output mapping, overlooking the importance of explicitly inducing and reasoning over such rules, limiting both performance and interpretability. We propose ERIM, Explicit Rule-Induction for Multimodal In-Context Learning, a unified framework that enhances MICL by explicitly inducing the latent rules governing demonstration samples. Leveraging the reasoning ability of multi-modal large language model (MLLM), ERIM constructs rule-centric MICL datasets, introduces explicit supervision for rule induction, and fine-tunes MLLM to perform explicit rule-induction via a two-stage training strategy that combines mixed cold-start supervised fine-tuning with reinforcement learning. Extensive experiments show consistent performance improvements across diverse MICL settings and demonstrate that correctly inducing underlying rules directly leads to stronger and more stable performance.