这个研究把视觉探索和符号规划结合起来,让模型在虚拟环境里干活成功率超90%,比直接让大模型操作强多了,而且不用专门训练。
该研究提出一种神经符号智能体,将长时程家务任务分解为任务导向的视觉探索和受限符号规划。第一阶段,视觉语言模型和探索机制从第一视角观察中获取目标相关谓词和实例绑定,生成符号初始状态。第二阶段,PDDL转移模型限制解码,蒙特卡洛树搜索评估可执行延续。在VirtualHome和ALFWorld上,4B-27B模型成功率超90%,最小智能体显著优于27B直接视觉策略。约束与搜索互补,组合解决超95%任务,且生成token更少。
Neurosymbolic Embodied Agents
Language and vision-language models generate plausible embodied plans but do not guarantee executability, as their outputs can violate environment dynamics or act on incorrectly grounded entities. We present a neurosymbolic agent that factors long-horizon household tasks into task-directed visual exploration and constrained symbolic planning. In the first phase, a vision-language model and exploration harness acquire goal-relevant predicates and instance bindings from egocentric observations and grounded interactions, producing a symbolic initial state. In the second, a PDDL transition model restricts decoding to tokens that extend applicable actions. Monte Carlo tree search then evaluates executable continuations using a domain-independent planning heuristic. The resulting plans are executable by construction under the transition model, with transfer to the environment conditioned on correct visual grounding. On VirtualHome and ALFWorld, open 4B-27B models exceed 90% success in both environments, and our smallest agent substantially outperforms a 27B direct visual policy in each. Constraints and search prove complementary rather than interchangeable: in ALFWorld either alone solves under a third of tasks, whereas their combination solves over 95%. The method also uses several times fewer generated tokens than extended thinking and far fewer model-visible images than direct interaction, and residual failures localize to state acquisition rather than plan generation without any specialized training.