Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models

精选理由

RuleMaze提出了一个创新的基准,通过DMP技术显著提升了MLLMs的规则遵循能力,对于研究MLLMs的空间规划具有重要意义。

AI 摘要

RuleMaze是一个可控基准,要求MLLMs在迷宫中遵守自然语言规则。通过自动生成自然语言规则并将其转换为逻辑表示,RuleMaze消除了手动规则工程的需要。Disentangled Multimodal Planning (DMP)通过可解释的推理原语分离感知、执行和规则验证,提高了规则遵循和泛化能力。实验表明,DMP在规则遵循和规划成功率方面显著优于端到端文本规划基线。

原文 · arXiv cs.AI

Multimodal large language models (MLLMs) combine linguistic reasoning with visual perception, yet their ability to perform visual spatial planning under explicit or previously unseen rule constraints remains underexplored. This setting requires models to jointly understand spatial layouts, interpret natural-language rules, and plan valid actions accordingly. To address this gap, we introduce RuleMaze, a controllable benchmark in which MLLMs must navigate mazes while obeying natural-language rules of varying complexity. RuleMaze isolates rule-compliant spatial planning by requiring accurate perception, rule interpretation, and constrained action planning. To enable scalable and systematic rule construction, we propose Language-Logic-Function Hybridization, which automatically generates natural-language rules and translates them into logical representations and executable validators, eliminating manual rule engineering. To improve rule following and generalization, we introduce Disentangled Multimodal Planning (DMP), which separates perception, execution, and rule verification through interpretable reasoning primitives. By disentangling these components, DMP facilitates systematic generalization to more complex and previously unseen rules, while providing transparent intermediate planning traces. Experiments demonstrate that DMP substantially improves rule compliance and planning success compared to end-to-end textual planning baselines. Overall, RuleMaze establishes a principled benchmark for studying grounded and interpretable rule-based spatial planning in MLLMs. Code is available at https://github.com/oceanflowlab/RuleMaze.