论文精选

MindTopo 基准测试模型发布,评估大模型拓扑空间推理能力

MindTopo: Can Foundation Models Reason in Topological Space?

精选理由

想了解大模型在拓扑空间推理方面的真实能力,可以看看这个新发布的 MindTopo 基准测试,它专门评估了 14 款模型的表现。

研究人员发布 MindTopo 基准测试,评估 14 款多模态大语言模型(如 Qwen3-VL-2B-Instruct)在拓扑空间推理(连续性、分离性等)和规划任务上的表现。结果显示模型在推理任务上表现优于规划任务,但均远低于人类水平。

原文 · arXiv cs.AI

MindTopo: Can Foundation Models Reason in Topological Space?

Spatial reasoning depends not only on metric properties such as distance, angle, and shape, but also on topological relations that remain invariant under continuous deformation. Cognitive science identifies these relations as foundational to spatial understanding, yet foundation-model evaluations largely focus on metric or viewpoint-dependent relations. We introduce MindTopo, a benchmark of topological intuition across five properties grounded in cognitive science and formal topology: continuity, separation, order, enclosure, and knots. MindTopo evaluates each property at two cognitive levels. Reasoning asks a model to identify topological relations or infer how they change. Planning instantiates a foundation model as a closed-loop agent whose policy selects environment actions. MindTopo contains 11,030 instances across 13 procedurally generated task types with controllable difficulty. We benchmark 14 MLLMs and study agent configurations augmented with image and video generation, including 3 video generative models in planning settings. Every MLLM performs better on reasoning than on planning, and the best-performing model remains far below observed human performance. On Qwen3-VL-2B-Instruct, supervised fine-tuning and reinforcement learning improve reasoning more than planning. Generated observations retain local cues and reach plausible endpoints, but audited rollouts do not reliably follow environment dynamics or preserve topology across transitions. Our website is at https://mind-topo.github.io/