做多模态模型可解释性研究的团队，终于有了一个能量化模态间协同与冗余的框架，比传统的表示对齐和结果评估更深入，值得关注其方法论和实验设计。
本文提出使用部分信息分解（PID）作为决策级框架，用于分离多模态大语言模型中视觉和语言输入的独特、冗余和协同贡献。在视觉-语言基准测试中，PID揭示了重复出现的模态使用模式：推理和接地任务表现出高协同性，而专家和知识任务则更依赖语言独有信息。这些模式在不同模型家族中泛化，并能预测对模态级干预的敏感性。研究还扩展了PID到三模态系统（感官PID），将语言作为控制变量分解视频-音频信息增益，发现即使在音视频融合任务中，视觉信息也主导了感官协同瓶颈。最后，PID引导的重新加权初步证明了改善多模态推理和接地性能的潜力。
Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition
Understanding modality interaction in multimodal large language models (MLLMs) is central to reliable deployment. We introduce Partial Information Decomposition (PID) as a decision-level framework that separates unique, redundant, and synergistic contributions of sensory and linguistic inputs, beyond representation alignment and outcome-based evaluation. Across vision--language benchmarks, PID reveals recurring modality-use profiles: reasoning and grounding-oriented tasks tend to exhibit high synergy, whereas expert and knowledge-oriented tasks show stronger language-unique reliance. These profiles generalize across model families and predict sensitivity to modality-level interventions. We further extend PID to tri-modal systems with Sensory PID, treating language as a control variable to decompose video--audio information gain. Applied to omni-modal models, Sensory PID reveals a sensory synergy bottleneck dominated by visual information even on audio--visual fusion tasks. Finally, PID-guided reweighting provides initial evidence for improving multimodal reasoning and grounding performance.