Papers

arXiv 论文提出 PAPAV 框架,统一评估从电脑操作到机器人的具身智能体

Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use

Why we picked it

这篇论文把电脑操作智能体和机器人放在同一套 PAPAV 五能力框架里比较,还统计了 64 个基准里验证和预判环节几乎没人评,做具身方向的人可以对照看看自己的评估盲区。

arXiv 论文 2610.30108 定义了多模态具身智能体(MMEAs),即把多模态任务推理与物理行动耦合、并根据反馈修正决策的系统。论文提出 PAPAV 框架,用五个能力维度刻画这类系统:感知状态、预判动作效果、规划路径、执行操作、验证结果。作者基于 64 个基准的统计发现,评估集中在 Act 环节,只有 5 个基准评估 Anticipate、3 个评估 Verify。论文还归纳了从部分可观测性到不可验证结果在内的五项物理约束,说明机器人场景比电脑操作场景更难回退错误。

Original · arXiv cs.AI

Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use

Multimodal agents (MMAs) sustain interaction through reasoning, memory, tools, and feedback, most visibly as computer-use agents, while robotic systems (RSs) couple sensing and actuation under physical dynamics. We define multimodal embodied agents (MMEAs) as goal-directed systems that couple multimodal task reasoning with physical action and revise decisions from the resulting feedback. This raises a central question: what changes when a multimodal agent moves from computer-use to robot-use? We introduce PAPAV, a unified capability-centric framework of five recurring functions: Perceive the current state, Anticipate action effects, Plan a feasible course, Act through an interface or body, and Verify the outcome. Defined by function rather than architecture, these capabilities let PAPAV identify what all three share and then compare MMEAs with MMAs and with RSs. Five physical constraints, from partial observability to unverifiable outcomes, leave fewer choices fixed in advance and less room to reverse errors. Across 64 benchmarks, explicit evaluation centers on Act; only 5 assess Anticipate and 3 assess Verify, leaving both largely hidden behind task success. These constraints also frame the open challenges we identify. PAPAV therefore provides a common basis for designing reliable physical agents and measuring progress across all five capabilities rather than task success alone.