AnchorSim 论文指出多模态潜在推理存在心智模拟坍缩问题
Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning
这篇论文做了个很扎心的实验：把模型的视觉 latent 换掉或删掉，答案几乎不变，说明所谓视觉推理其实是文本在干活。作者提出 AnchorSim 对比蒸馏来修复这个问题，做潜在推理方向的研究者值得对照一下自己的训练目标。
arXiv 论文（编号 2611.10423）通过对多个潜在推理 MLLM 的扰动分析发现，生成式视觉潜在块在最终答案中往往不起作用：丢弃、替换或打乱这些视觉 latent 对答案概率影响很小，而对文本推理链做同等扰动则损害明显得多。作者将这种失败模式命名为 Mental Simulation Collapse（MSC），即模型看似在做交错式视觉推理，实际上把与答案相关的计算都路由到了文本通路。
论文进一步诊断出 MSC 的成因来自现有训练目标：视觉对齐只能让生成的 latent 接近辅助图像的嵌入，但不要求解码器真正依赖它们，随后的纯文本监督又使视觉通路的语义角色无法被识别。为此论文提出 AnchorSim（Anchored Mental Simulation），一个对比蒸馏框架，训练模型让生成的视觉 latent 对预测产生实际影响。
在多个视觉推理基准上，AnchorSim 提升了性能，诊断性复评显示视觉扰动现在能明显改变模型预测。这一发现对正在构建潜在多模态推理的研究者意味着：仅靠对齐目标不足以让内部视觉工作空间真正参与计算。
Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning
Mental simulation, the ability to reason by constructing and updating internal visual states, is central to flexible human cognition, but remains elusive in current multimodal large language models. Latent multimodal reasoning offers a promising route toward this capability by interleaving text with continuous visual latent blocks, allowing models to maintain internal visual workspaces without decoding them into pixels. In this paper, we ask whether these latent visual states actually matter. Through a direct perturbation analysis across representative latent-reasoning MLLMs, we find that they often do not: dropping, replacing, or shuffling generated visual latents has only minor effects on the model's own final-answer probability, while equivalent perturbations to the linguistic reasoning trace are substantially more damaging. We call this failure mode Mental Simulation Collapse (MSC): the model appears to perform interleaved visual reasoning, but routes most answer-relevant computation through text. We further diagnose that MSC follows naturally from existing training objectives: visual alignment can make generated latents resemble helper-image embeddings, but it does not require the decoder to rely on them; subsequent text-only supervision further leaves the semantic role of the latent pathway unidentifiable. To address this, we introduce AnchorSim (Anchored Mental Simulation), a contrastive distillation framework that trains models to make their generated visual latents consequential. Across multiple visual reasoning benchmarks, AnchorSim improves performance, while diagnostic re-evaluation shows that visual perturbations now meaningfully affect predictions. Our results expose a fundamental failure mode of latent multimodal reasoning and demonstrate that mental simulation must be anchored to model behavior in order to matter.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include 多模态, AnchorSim, MLLM, 推理模型.
Source attribution: curated from arXiv cs.AI (@Mi Luo, Rohan Dugad, Zihui Xue, Alex Dimakis, Kristen Grauman).
