做具身智能、机器人或空间推理的团队,ESI-Bench 直接点出了当前 MLLM 在主动探索和行动决策上的致命短板,看完会重新思考你的感知-行动闭环设计。
研究团队提出 ESI-Bench,一个基于 OmniGibson 的具身空间智能基准,涵盖 10 类任务和 29 个子类,要求智能体主动探索而非被动感知。实验发现,主动探索显著优于被动多视角,但大多数失败源于“行动盲视”——错误行动导致错误观察,进而引发级联错误。即使引入 3D 空间理解,不完美的 3D 表示反而比 2D 基线更差。人类研究显示,模型缺乏元认知能力,过早下结论且不修正信念,暴露了感知与行动闭环中的关键鸿沟。
ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop
Spatial intelligence unfolds through a perception-action loop: agents act to acquire observations, and reason about how observations vary as a function of action. Rather than passively processing what is seen, they actively uncover what is unseen - occluded structure, dynamics, containment, and functionality that cannot be resolved from passive sensing alone. We move beyond prior formulations of spatial intelligence that assume oracle observations by recasting the observer as an actor. We introduce ESI-BENCH, a comprehensive benchmark for embodied spatial intelligence spanning 10 task categories and 29 subcategories built on OmniGibson, grounded in Spelke's core knowledge systems. Agents must decide what abilities to deploy - perception, locomotion, and manipulation - and how to sequence them to actively accumulate task-relevant evidence. We conduct extensive experiments on state-of-the-art MLLMs and find that active exploration substantially outperforms passive counterparts, with agents spontaneously discovering emergent spatial strategies without explicit instructions, while random multi-view often adds noise rather than signal despite consuming far more images. Most failures stem not from weak perception but from action blindness: poor action choices lead to poor observations, which in turn drive cascading errors. While explicit 3D grounding stabilizes reasoning on depth-sensitive tasks, imperfect 3D representation proves more harmful than 2D baselines by distorting spatial relations. Human studies further reveal that unlike humans who seek falsifying viewpoints and revise beliefs under contradiction, models commit prematurely with high confidence regardless of evidence quality, exposing a metacognitive gap that neither better perception nor more embodied interaction alone can close.