OPD-V:基于模态平衡的视觉在线自蒸馏

OPD-V: Visual On-Policy Self-Distillation with Modality Balance

精选理由

OPD-V把模态平衡当特权信息,用正负教师挑token做自蒸馏,在6个基准、4种骨干上更省训练费还更准。

AI 摘要

现有在线自蒸馏方法忽视多模态大模型推理中的模态失衡问题,文本信息占优时多模态输入无法充分整合。OPD-V利用放大图像的正面教师与遮罩图像的负面教师,将模态平衡本身作为特权信息。通过正负教师的模态平衡对数边际定义信任区域,筛选在线策略token用于自蒸馏。在6个基准、4个MLLM骨干和5种后训练方法上,OPD-V一致提升推理性能并降低训练成本。

原文 · arXiv cs.AI

OPD-V: Visual On-Policy Self-Distillation with Modality Balance

On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (MLLMs). Existing methods draw privileged information from diverse input sources to guide self-distillation. Yet these designs overlook Modality Imbalance, a challenge inherent to MLLM reasoning. When textual information dominates generation, the model cannot fully integrate its multimodal input. Consequently, carefully designed privileged information remains underused, limiting the effectiveness of OPSD. To examine this limitation, we construct a Positive Teacher with the Zoom-In Image and a Negative Teacher with the Mask Image, which exhibit different degrees of Modality Imbalance. Changes in their reasoning correctness and token logits reveal that Modality Balance can itself serve as privileged information. Motivated by this finding, we introduce OPD-V, a visual OPSD paradigm that instantiates such information through the Positive Teacher and Negative Teacher. Positive Modality-Balance Logits Margins define a Modality-Balance Trust Region that selects the on-policy tokens used for self-distillation. Experiments across 6 benchmarks, 4 MLLM backbones, and 5 post-training methods show that OPD-V consistently improves reasoning performance while reducing training cost.