VAORA 用一个巧妙的奖励设计,让 VLM 在物理推理中不再瞎想,在 PHYRE 和 Virtual Tool 上泛化能力更好。
VAORA (Visual Action Outcome Reasoning Alignment) 提出两种互补奖励:Visual Alignment Reward 将 VLM 推理锚定到视觉上下文,独立于智能体动作;Visual-Action Alignment Reward 将推理与动作引起的视觉结果对齐。该方法在 PHYRE 和 Virtual Tool 基准上,针对未见任务和未见环境设置提升性能,抑制幻觉链式推理并缩小推理与行为差距。实验表明,VAORA 能诱导基于视觉的、可泛化的物理智能。
Bridging Physical Reasoning and Task Generalization via Visual Action Outcome Reasoning Alignment
Vision-language models (VLMs) struggle to generalize in interactive physical reasoning, particularly under unseen tasks and environments. Two key failure modes are prominent: hallucinated chain-of-thought (CoT) reasoning that contradicts physical reality, and misalignment between the model's reasoning and actions. We present VAORA (Visual Action Outcome Reasoning Alignment), a novel reward design that directly addresses both issues. VAORA introduces two complementary rewards: Visual Alignment Reward, which anchors VLM reasoning to the visual context independent of the agent action itself, and Visual-Action Alignment Reward, which grounds reasoning in the visual outcome induced by the model's action. Together, these rewards suppress hallucinated CoT and reduce the gap between reasoning and behavior. To improve training stability, we further employ smooth, dense rewards by estimating success probabilities using a pre-trained in-domain expert agent. Experiments on PHYRE and Virtual Tool support our performances across novel-task and unseen-environment settings, confirming that grounded and generalizable physical intelligence can be induced through VAORA.