这篇论文用无标签数据先学移动再学做事,SIMPLER上提升10%,真实机器人也抗干扰,思路很实用。
VLA模型受限于专家演示数据稀缺,这些数据需要观测、指令和动作的三元组,成本高昂。研究提出分解假设,将物理能力(如何移动)与语义对齐(做什么)解耦。基于此设计了TAP框架,先通过自监督逆动力学从无标签交互数据学习运动先验,再用少量专家数据将先验与语言对齐。在SIMPLER基准上,TAP匹配使用超过100万专家轨迹的模型,实现了10%的绝对提升。真实WidowX机器人平台测试中,TAP在相机扰动下保持25%成功率,而互联网规模基线降至0%。
Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
Vision-Language-Action (VLA) models are fundamentally bottlenecked by the scarcity of expert demonstrations -- triplets of observations, instructions, and actions that are costly to collect at scale. We argue that this bottleneck stems from conflating two distinct learning objectives: acquiring physical competence (how to move) and acquiring semantic alignment (what to do). Crucially, only the latter requires language supervision. Building on this Decomposition Hypothesis, we propose Task-Agnostic Pretraining (TAP), a two-stage framework that first learns transferable motor priors from cheap, unlabeled interaction data -- including discarded off-task trajectories and autonomous robot play -- via a self-supervised Inverse Dynamics objective. A lightweight second stage then grounds these priors in language using minimal expert data. On the SIMPLER benchmark, TAP matches models trained on over 1M expert trajectories while using orders of magnitude less labeled data, yielding a 10% absolute gain over standard behavior cloning. On a real-world WidowX platform, TAP retains 25% success under camera perturbations where internet-scale baselines collapse to 0%, demonstrating that task-agnostic pretraining produces robust, transferable physical representations and offers a scalable path forward for Embodied AI.