做多模态智能体或空间推理研究的团队，这个基准直接暴露了当前模型在真实交互任务上的短板——GPT-5都只有17.4%成功率，值得用来检验自家模型。
多模态大模型在物理世界中的空间推理能力至关重要，但现有基准多依赖静态问答或特定模拟器，无法评估真实交互场景。研究者提出SpatialWorld，一个统一基准，整合8种异构仿真后端，包含760个人工标注任务，覆盖家务、旅行、社交协作等领域。智能体需在仅视觉部分可观测条件下主动收集证据，并通过统一文本接口做出决策。评估15个先进智能体发现，最强模型GPT-5平均任务成功率仅17.4%，开源模型Qwen-3.5为14.1%，表明主动探索和长程规划仍是瓶颈。
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
Spatial reasoning is a foundational capability for multimodal large language models (MLLMs) to perceive and operate within the physical world. However, existing benchmarks predominantly rely on passive evaluation (e.g., static VQA) or simulator-specific pipelines, failing to assess general interactive spatial understanding. We introduce SpatialWorld, a unified benchmark designed specifically for evaluating the interactive spatial understanding of multimodal agents in complex real-world tasks. Integrating eight heterogeneous simulation backends under a shared, simulator-agnostic protocol, SpatialWorld features 760 human-annotated tasks across diverse domains (e.g., household routines, travel, social collaboration). Agents must solve tasks under vision-only partial observability, actively gathering egocentric visual evidence and expressing decisions via a unified, text-based action interface native to MLLMs. For reliable evaluation, each task includes a human-validated initial state, a reference trajectory, and a terminal-state verifier. Evaluating 15 advanced agents reveals that robust spatial task solving remains challenging: the strongest model, GPT-5, achieves an average task success rate (TSR) of only 17.4%, while the leading open-source model, Qwen-3.5, reaches 14.1%. Further analysis exposes a clear mismatch between task success and execution efficiency, alongside substantial domain-specific performance variations. These bottlenecks in active exploration and long-horizon planning position SpatialWorld as a rigorous testbed for future spatial agents.
- AK06-09 16:09原文