这篇论文提出了一种新思路:用几何基础模型直接做机器人操作策略,不依赖二维图像,效果更快更准更轻。
Geometric Action Model (GAM) 将预训练的几何基础模型 (GFM) 拆分为观测编码器和未来预测解码器两部分,在中间层插入因果未来预测模块,结合语言、本体感受和动作历史,预测未来潜在标记,再利用剩余 GFM 块进行特征传播与动作解码。在多个仿真和真实机器人操作基准上,GAM 的准确率、鲁棒性、速度和模型大小均优于当前基于基础模型尺度的基线方法,例如在 RoboMimic 和 ManiSkill2 任务中表现显著提升。
Geometric Action Model for Robot Policy Learning
Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models, but they still operate primarily on 2D image frames or 2D-derived latent spaces, leaving implicit the 3D geometry required for contact-rich manipulation. We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy that directly repurposes a pretrained geometric foundation model (GFM) as a shared substrate for perception, temporal prediction, and action decoding. GAM splits the GFM at an intermediate layer: the shallow layers serve as an observation encoder, and a causal future predictor inserted at the split layer forecasts future latent tokens conditioned on language, proprioception, and action history. The predicted future tokens are then routed through the remaining GFM blocks for feature propagation and decoding, allowing a single backbone to produce both future geometry and actions. This design equips the GFM with language-conditioned temporal world modeling through minimal architectural modification while preserving its rich geometric priors. Across a broad suite of simulation and real-robot manipulation benchmarks, GAM is more accurate, more robust, faster, and lighter than current foundation-model-scale baselines.