SceneBind 把场景的“是什么”和“在哪里”一起建模，场景和空间检索都做到了 SOTA，还能零样本搞定音视频定位。
SceneBind 是一种全模态表示方法，通过联合语义和 3D 空间理解来建模真实场景。它使用全局语义嵌入与对象中心语义-空间槽，显式编码对象级语义、空间属性和不确定性。SceneBind Matching 方案整合全局场景相似性与对象对齐，支持跨模态场景检索和对象定位。在场景检索和空间检索基准上达到 SOTA，并在音频-视觉定位任务上实现零样本迁移。
SceneBind: Binding What and Where Across Vision, Audio and Language
We present SceneBind, an omni-modal representation of realistic scenes with joint semantic and 3D spatial understanding across vision, audio and language. Existing omni-modal encoders excel at instance-level semantics (i.e., what is present), but often lack explicit spatial structure (i.e., where it is). SceneBind addresses this gap by representing each scene as a semantic-spatial entity, combining a global semantic embedding with object-centric semantic-spatial slots. This representation explicitly captures object-level semantics, spatial attributes, and uncertainty. We further propose SceneBind Matching, a semantic-spatial matching scheme that integrates global scene similarity with object alignment, supporting cross-modal scene retrieval and object grounding. To train and evaluate SceneBind, we curate a novel real-world binaural audio-visual dataset with structured semantic and spatial annotations, and propose a training protocol for aligning semantic and spatial signals across modalities. SceneBind is compatible with large-scale pretrained semantic encoders, adds lightweight spatial modeling with only a few additional tokens. It achieves state-of-the-art scene and spatial retrieval while enabling strong zero-shot transfer to downstream tasks such as audio-visual localization.