做视觉AI的开发者会关心：你的ViT模型真的理解“蓝色圆形”是一个整体吗？这篇论文给出了量化绑定信息的方法，值得点开看看如何诊断模型的结构性盲点。
该论文从信息论角度形式化了视觉中的“绑定问题”，即系统如何知道哪些特征（如颜色、形状）属于同一个物体。研究者提出了一种探测方法，用于测量深度学习模型（尤其是Vision Transformers）内部表示中的绑定信息。实验发现，ViT的不同组件（如[CLS]标记和空间标记）包含不同程度的绑定信息，且在特征共享、遮挡等挑战性场景中表现差异显著。这项工作表明，绑定信息是强视觉识别与推理的关键要素，但目前模型仍存在特征误归因的常见失败。
Formalizing the Binding Problem
Representations of the world, arguably, contain information about features (e.g. something is blue, something is a circle) but also information about which features are part of the same object (e.g. the circle is blue), which we call binding information. Any system with the ability to understand scenes with multiple objects must be able to solve the binding problem: it needs to know which features belong together. However, despite work showing that Vision Transformers (ViTs) know which patches belong together, it is not known whether current deep learning models learn to exhibit binding information, i.e., for features. We may believe that there is not much binding information, after all misattributing features to wrong objects is a common failure of ViT-based architectures, especially in scenes with objects sharing features. Here we formalize the binding problem with an information-theoretic approach, and introduce a probing method to measure binding information in model representations. We perform experiments on ViTs, measuring binding from different components of the architecture, such as the image summary token [CLS] or the spatial tokens. We use datasets with different binding challenges, such as feature sharing, occlusion, and natural features, while comparing the performance of several pre-trained ViTs. Overall, our research demonstrates binding as a key ingredient to strong visual recognition and reasoning.