CoorDex让人形机器人在行走时用灵巧手完成开冰箱、抓瓶子的连续操作,不再需要走走停停,和之前的笨拙模式完全不一样。
CoorDex是一个学习管线,将高维全身和灵巧手控制转换为协调的潜在残差控制,使Unitree G1人形机器人搭配20-DoF WUJI手能在移动中执行灵巧操作。它从模拟全身和手部演示训练特权运动跟踪教师,蒸馏为潜在先验,再用残差强化学习训练协调策略。该方法实现了非停止的瓶子抓取搬运、移动中打开冰箱门和立方体旋转。消融实验表明,联合空间PPO、联合空间手控制和整体潜在预测在同一奖励预算下均失败。
CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulation
Humanoid loco-manipulation is often simplified into a stop-and-go process: walking to an object, stopping to manipulate it, and then resuming locomotion. It also commonly relies on low degree-of-freedom (DoF) end effectors that behave like an open-close grasp primitive. We introduce CoorDex, a learning pipeline that converts high-dimensional body and dexterous hand control into coordinated latent residual control, enabling high-DoF dexterous loco-manipulation on the move. Starting from simulated whole-body and hand demonstrations, CoorDex trains privileged motion tracking teachers for the humanoid body and dexterous hand, distills them into proprioception-conditioned latent priors, and uses the frozen priors as the action space for downstream residual reinforcement learning. A coordinated latent residual policy composes these priors through shared task context and separate body-hand residual heads, preserving natural whole-body motion while improving finger-level contact reliability. CoorDex enables a Unitree G1 humanoid with a 20-DoF WUJI hand to execute dexterous manipulation while in motion, including non-stop bottle grasping and carrying, fridge door opening on the move, and cube pick-and-turn. Ablations on the walk-grasp-carry task show that joint-space PPO, joint-space hand control, and monolithic latent prediction all fail under the same reward budget, while the latent-prior interface and coordinated residual structure make high-dimensional contact-rich loco-manipulation trainable. Project Page: https://skevinci.github.io/coordex/