这项研究戳破了全模态模型“感知接地”的泡沫——模型明明“看到”了矛盾却选择不说，做多模态推理或安全对齐的团队值得关注，尤其是音频模态的短板需要优先补上。
研究发现全模态大模型（如Gemini 3.1 Pro）在感知与行动之间存在显著鸿沟：模型内部隐藏状态能正确编码感知与文本前提的矛盾，但输出时几乎从不拒绝错误前提。研究团队构建了IMAVB基准测试，包含500个长视频片段，交叉测试视觉/听觉模态和标准/误导前提。模型表现出两种失败模式：欠拒绝（接受错误前提）和过拒绝（连正确前提也拒绝）。音频模态的接地能力弱于视觉，且该问题对七种提示变体均不敏感。提出的探针引导logit调整（PGLA）方法能有效改善拒绝行为，表明瓶颈在于从感知到行动的转化，而非感知本身。
Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMs
When an omnimodal large language model accepts a question whose textual premise contradicts what it actually sees or hears, does the failure lie in perception or in action? Recent omnimodal models are positioned as perception-grounded agents that jointly process video, audio, and text, yet a basic form of grounding remains untested: catching a textual claim that conflicts with the model's own sensory input. We introduce IMAVB, a curated 500-clip benchmark of long-form movies with a 2x2 design crossing target modality (vision, audio) and premise condition (standard, misleading), which lets us measure conflict detection separately from ordinary multimodal comprehension. Across eight open-source omnimodal LLMs and Gemini 3.1 Pro, we document a Representation-Action Gap: hidden states reliably encode premise-perception mismatches even when the same models almost never reject the false claim in their outputs. Behaviorally, models fall into two failure modes: under-rejection, in which they answer misleading questions as if the false premise were true; and over-rejection, in which they reject more often but also reject standard questions, sacrificing ordinary comprehension accuracy. The gap is modality-asymmetric (audio grounding underperforms vision) and prompt-resistant across seven variants. As an initial diagnostic intervention, a probe-guided logit adjustment (PGLA) re-injects the encoded mismatch signal into decoding and consistently improves rejection behavior. Together, these results suggest the bottleneck for omnimodal grounding lies in translation, not perception.