人形机器人操作新方法ROVE，用强化学习从糟糕的人类演示中挑出好动作，真实任务效果比基线好。
ROVE 是一个用于人形机器人视觉-语言-动作（VLA）模型后训练的强化学习框架，能够应对不完美的人类干预数据。它引入人类在环流水线收集部署与干预数据，并使用乐观价值估计（OVE）从混合质量轨迹中筛选高价值行为。ROVE 还利用跨实体人类经验视频为长尾失败与恢复模式提供丰富监督，引导 VLA 聚焦于高价值行为。在真实世界的接触丰富且精细的人形操作任务中，ROVE 超越了经验学习基线，并在多轮部署-干预迭代中持续提升。
ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement Learning
Human interventions provide crucial corrective signals for post-training Vision-Language-Action (VLA) models. However, enabling seamless humanoid interventions is a formidable systems challenge due to complex whole-body kinematics and dexterous-hand control. Consequently, the collected intervention trajectories are often suboptimal, and methods that rely on human interventions as expert supervision can absorb hesitant, inefficient, or even erroneous behaviors. To address both the system and algorithmic challenges, we propose ROVE, a reinforcement learning framework for humanoid VLA post-training with imperfect human interventions. First, ROVE introduces a human-in-the-loop pipeline capable of collecting deployment and intervention data for humanoid manipulation. Second, it utilizes Optimistic Value Estimation (OVE) to prioritize high-value behaviors from mixed-quality trajectories. To further robustify value estimation, we incorporate cross-embodiment human experience videos to provide rich supervision for long-tailed failure and recovery modes. The resulting critic yields informative advantage signals, steering the VLA actor to focus on high-value behaviors rather than indiscriminately imitating all actions. On challenging real-world contact-rich and fine-grained humanoid manipulation tasks, ROVE outperforms experience-learning baselines and consistently improves across multiple rollout-intervention iterations.