跨实体机器人操作的动作先验学习

Learning Action Priors for Cross-embodiment Robot Manipulation

精选理由

这篇论文教VLA模型在正式训练前先学动作规律,用的是流匹配,13个任务上效果比直接训练好,数据少时尤其明显。

AI 摘要

本文提出在两阶段框架中预训练动作模块,在VLA训练前注入运动先验。阶段1使用基于流匹配的轻量级编解码器,仅从无条件动作轨迹学习跨实体时间运动结构,无需处理视觉或语言token。阶段2通过解码器重用和早期潜在蒸馏,将学习到的先验迁移至VLA训练,同时保留端到端优化。在13个仿真和真实世界的跨实体任务中,该方法比无先验的VLA训练收敛更快、成功率更高,尤其数据稀缺时表现更优。扩展阶段1的动作数据能提升下游VLA性能的泛化性。

原文 · arXiv cs.AI

Learning Action Priors for Cross-embodiment Robot Manipulation

Most Vision-Language-Action (VLA) models build on a Vision-Language Model (VLM) backbone by attaching an action module and optimizing the full policy jointly. This design inherits strong visual and linguistic priors from the VLM, but leaves the action module to learn physical motion almost from scratch. As a result, the policy lacks an explicit motion prior, forcing early optimization to simultaneously discover temporal action dynamics and cross-modal alignment, a challenge further amplified in cross-embodiment settings. In this work, we propose to pretrain the action module with motion priors before cross-modal VLA alignment. Specifically, we introduce a two-stage training framework that equips the action module with cross-embodiment temporal motion structure before VLA training begins. In Stage~1, a lightweight flow-matching-based encoder-decoder action module efficiently learns temporal motion structure solely from unconditioned action trajectories, without processing visual or language tokens. In Stage~2, this learned prior is transferred to VLA training through decoder reuse and early-stage latent distillation, aligning visual-language features with the action embedding space while still allowing end-to-end policy refinement. In addition, the trained encoder serves as a compact history compressor, summarizing state-action histories into a single temporal context token for history-aware modeling at negligible cost. Extensive experiments across 13 diverse cross-embodiment tasks on both simulated and real-world platforms validate the effectiveness of our approach. Compared with VLA training without action priors, our model achieves faster convergence, higher success rates, and substantially stronger performance on data-scarce real-world tasks. Moreover, scaling up the action data in Stage~1 yields a more generalizable action prior that directly improves downstream VLA performance.