RECALL:VLA模型的主动持续学习恢复经验收集

RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models

精选理由

这篇论文研究了怎么让机器人在学新任务时不忘旧技能,用不确定性主动挑数据微调VLA模型,比被动收集更高效,还试了两种防遗忘方法,挺实在的。

AI 摘要

论文提出RECALL方法,用于视觉-语言-动作(VLA)模型的主动持续学习。与被动模仿学习相比,不确定性引导的数据收集使微调效率提升30%以上。但仅训练恢复数据会导致灾难性遗忘,在OpenVLA模型上丢失20%的旧任务性能。评估了重放混合和弹性权重巩固(EWC)两种持续学习技术,发现可塑性与记忆保留之间存在权衡。实验在3个机器人操作任务上进行,表明不确定性引导的恢复演示能提升适应效率,但如何平衡新旧知识仍是开放挑战。

原文 · arXiv cs.AI

RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models

Vision-Language-Action (VLA) models are commonly fine-tuned through passive imitation learning, where additional demonstrations are collected for tasks where the policy performs poorly. This approach incurs several downsides: it requires the robot to fail before data collection is triggered, provides little guidance about which states require supervision, and wastes demonstrator effort on redundant parts of the task where the policy already performs well. In this paper, we propose an active, continual learning paradigm for VLAs. We demonstrate that active, uncertainty-guided data collection leads to more efficient fine-tuning than when using passively-collected demonstrations. However, we also find that fine-tuning only on actively-collected recovery data leads to catastrophic forgetting. We evaluate techniques for continual learning, including replay-based data mixing and elastic weight consolidation, and identify tradeoffs between plasticity to uncertainty-guided recovery data and retention of previously learned behaviors. Overall, our work contributes an empirical study of active continual learning for autoregressive VLAs, establishing that uncertainty-guided recovery demonstrations can improve adaptation efficiency while also revealing open challenges when targeted new data is incorporated into large robot policies.