想搞懂多模态模型内部怎么运作？MMDiff能找出是哪些特征让模型变聪明，还能精准控制它们，安全性和效果都有数据支撑。
MMDiff是一个多模态模型差分框架，通过训练多模态稀疏自编码器（SAE）来发现和控制多模态大模型（MLLM）的内部特征。该框架支持特征隔离、任务特定特征检测和特征级控制三种用途。研究团队在LLaVA-MORE、PaliGemma 2和InternVL3.5三个模型家族上训练了多模态SAE，并在视觉空间理解、多模态安全和OCR任务上进行了评估。移除MMDiff发现的特征可使空间任务性能平均下降12%、OCR下降17%，多模态安全攻击成功率降低24%，且不影响VQA性能。特征引导在空间和OCR任务上分别比标准单层引导基线平均提升+3.6%和+1.8%。
Multimodal Model Diffing for Feature Discovery and Control
Multimodal Large Language Models (MLLMs) exhibit strong visual understanding, yet the internal features that cause these behaviors remain difficult to identify, audit, or control. While applicable to post-hoc inspection, hidden states that are decomposed into interpretable feature directions using sparse autoencoders (SAEs) neither readily isolate which features are changed by multimodal training, nor are they directly useful for targeted control. We introduce MMDiff, a multimodal model-diffing framework that trains multimodal SAEs and turns them into feature-level interfaces for discovering and controlling multimodal behavior. MMDiff supports three uses: (i) feature isolation, by diffing a base-LM SAE against its multimodal-adapted counterpart to identify features altered by multimodal training; (ii) task-specific feature detection, via per-token contrastive firing analysis that isolates causal features; and (iii) feature-level control, by causally removing or steering the discovered feature directions. We train multimodal SAEs for three MLLM families, LLaVA-MORE, PaliGemma 2, and InternVL3.5, and evaluate on visual-spatial understanding, multimodal safety, and OCR. MMDiff discovers sparse, causally specific features whose removal selectively degrades target behaviors by an average of 12% on spatial tasks and 17% on OCR, and reduces attack success rate by 24% on multimodal safety attacks, with no impact on VQA performance. Steering these features improves spatial and OCR accuracy by +3.6% and +1.8% on average over a standard single-layer steering baseline. These results show that multimodal SAEs can serve not only as interpretability tools, but as mechanisms for auditing, steering, and controlling MLLMs behavior toward safer and more capable generations.