这篇论文提出了InSight框架,让机器人通过VLA模型自己学新技能,不用人教,就能搞定方块翻转、倒水这些操作,很有实用性。
InSight框架通过将VLA模型在原始动作层面变得可操控,从而解锁自主技能获取能力。该框架包含自动分割管道,利用VLM计划分解和末端执行器姿态将演示分割成带标签的原始动作,以及VLM引导的数据飞轮,自动识别缺失原始动作并尝试演示。在模拟和真实世界操作任务中(包括方块翻转、抽屉关闭、清扫、扭转、倒水)评估,无需任何人工演示即可学习这些技能。学到的原始动作可组合执行新颖的长周期任务,无需额外人工演示。
InSight: Self-Guided Skill Acquisition via Steerable VLAs
Vision-language-action (VLA) models can learn manipulation skills from demonstrations, but their capabilities are bounded by the skills in the training data. We present InSight, a framework that unlocks autonomous skill acquisition by rendering VLAs steerable at the primitive-action level (e.g., "move gripper to the bowl", "lift upward", "pour the bottle"). InSight consists of two primary stages: (1) an automated segmentation pipeline that partitions demonstrations into labeled primitives via VLM plan decomposition and end-effector poses to enable VLA primitive steerability, and (2) a VLM-guided data flywheel that identifies missing primitives required to accomplish a novel task, autonomously attempts demonstrations of the missing primitives with VLM-proposed low-level control, and automatically labels, stores, and integrates successful demonstrations into the VLA training set. We evaluate InSight across simulation and real-world manipulation tasks, including block flipping, drawer closing, sweeping, twisting, and pouring, without any human demonstrations of these target skills. Once learned, these primitives can be composed to execute novel, long-horizon tasks without additional human demonstrations. Our findings demonstrate that primitive steerability provides a practical foundation for continual skill acquisition in VLA policies. Project website: https://insight-vla.github.io.