想让人形机器人自己学会拿东西？VLK用48,000条合成轨迹模拟真实场景，直接在Unitree G1上跑通了导航和运输，省掉人工标注的苦力。
VLK提出利用3D高斯溅射（3D Gaussian Splatting）重建度量级室内场景，并合成导航与物体交互轨迹，生成48,000个配对视觉-语言-运动学（VLK）数据，无需人工干预。训练出的VLK策略能够预测短时全身运动轨迹，并通过全身跟踪器将预测转化为实际动作。在物理Unitree G1人形机器人上，该方法成功完成导航和单物体运输任务，实现了从模拟到真实的感知驱动操作。
VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes
Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion. Learning this mapping requires synchronized egocentric images, language commands, and robot-compatible kinematic trajectories, yet no existing data source provides this complete tuple at scale. We address this bottleneck by generating vision-language-kinematics (VLK) supervision synthetically in reconstructed scenes. Our pipeline leverages 3D Gaussian Splatting to reconstruct metric-scale indoor environments, synthesizes navigation and object-interaction trajectories using privileged scene information, and renders paired egocentric observations after the fact. We produce 48,000 paired trajectories with no human intervention and train a VLK policy that predicts short-horizon whole-body kinematic trajectories. A whole-body tracker converts these predictions into actions on the physical humanoid. We evaluate on the physical Unitree G1 performing navigation and single-object transport, demonstrating that synthesized interactions in reconstructed scenes provide effective supervision for sim-to-real perception-based humanoid loco-manipulation. Project Website: https://vision-language-kinematics.github.io/