Papers

V-lens:透过统一多模态模型的隐藏状态可视化推理概念

The Elephant in the Question: Unified Models Picture the Concepts Behind Their Answers

Why we picked it

给做多模态模型可解释性研究的人:这篇把隐藏状态直接画成图来看模型在想什么,不用训练解码器。lens tuning 把代词指代可视化率从 2-11% 拉到 86-94%,还顺带提升了文本答题,思路挺新。

arXiv 论文《The Elephant in the Question》提出 V-lens,无需训练解码器即可将统一多模态模型中文本 token 的隐藏状态可视化成图像,覆盖 4 个扩散架构与 2 个纯自回归的统一模型。其机制是利用统一架构中图像生成器可直接访问文本处理阶段表征这一特点,把中间概念画出来,即使这些概念在回答中从未被显式提及。

Body

论文进一步提出 lens tuning,用图片对选定的隐藏状态做监督。在 BAGEL、SenseNova-U1 和 HunyuanImage-3.0 上,代词指代对象的正确可视化率从 2-11% 提升到 86-94%,且这种视觉监督还能间接提高文本回答质量。受控实验展示了可视化内容随上下文、token 位置和深度的变化规律。

对研究者而言,这提供了一条检查模型推理中间步骤的可解释性路径,代码已在 GitHub 开源。

Original · arXiv cs.LG

The Elephant in the Question: Unified Models Picture the Concepts Behind Their Answers

Visualizing intermediate representations could help us inspect the reasoning steps underlying a model output. Unified multimodal models offer this opportunity by combining understanding and generation in one architecture. Their image generators can access representations formed while processing text. We introduce the V-lens, which uses this connection to visualize a chosen token's hidden state without training a decoder. Across four diffusion-based and two purely autoregressive unified models, the resulting images reveal key intermediate concepts that connect a question to its answer even when the concepts are never named explicitly. Controlled tests show how this content changes with context, token position, and depth. We then introduce lens tuning, which supervises selected hidden states with pictures through the V-lens. Across BAGEL, SenseNova-U1, and HunyuanImage-3.0, the rate of correct visualizations of pronoun referents rises from 2-11% to 86-94%. Surprisingly, such visual supervision also improves text answers without supervising the answer tokens directly. Code is available at https://github.com/multimodal-ai-lab/V-lens.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include V-lens, BAGEL, HunyuanImage-3.0, multimodal.

Source attribution: curated from arXiv cs.LG (@Tobias Braun, Marcus Rohrbach, Lorenzo Cavallaro).