阿里DAMO Academy开源的CamVLA,机器人不用提前知道相机位置也能适应视角变化,单张RGB图像搞定,部署更灵活。
CamVLA(Camera-Centric VLA)是一种新的视觉-语言-动作模型,通过预测相机坐标系中的末端执行器动作和6-DoF手眼矩阵,解耦了操作控制与相机几何。该方法无需相机外参标定、深度信息或立体视觉,仅需单目RGB图像即可适应任意视角变化。在模拟和真实机器人数据上的评估显示,CamVLA在多种未见过的视角下持续提高了任务成功率。
From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: https://alibaba-damo-academy.github.io/CamVLA/.