CSWAM模型提升机器人环境适应能力
CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models
这个模型对机器人很有用，能更好地适应新环境，比之前的FastWAM效果更好。
CSWAM通过因果语义专家改进了FastWAM，在视觉分布变化下表现更好。它使用V-JEPA 2.1学习语义状态变化，从稀疏历史中预测未来，并在动作去噪时结合当前视频状态和语义历史。在RoboTwin 2.0上，其随机化成功率从10.16%提升至45.18%，比FastWAM高出35.02个百分点。
CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models
FastWAM-style world action models enable efficient action-only inference, but generalize poorly under visual distribution shifts. Their reconstruction-oriented representations emphasize appearance-specific details, limiting generalization to unseen scenes and objects. Without observation history, the model also lacks temporal evidence for robustly identifying task-relevant state changes and motion in unfamiliar visual conditions. To address these limitations, we present the Causal Semantic World Action Model (CSWAM), which augments FastWAM with a causal semantic expert built on V-JEPA 2.1. V-JEPA provides temporally grounded representations of semantic state changes and motion with less dependence on appearance-specific details. The expert learns their future evolution from a sparse history of current and past observations and shares the history-derived context with both the video and action streams through causal attention. At inference, CSWAM conditions action denoising on the current video state and observed semantic history, retaining efficient action-only inference. We conduct simulation and real-robot experiments to evaluate generalization under distribution shifts. With embodied pretraining, CSWAM raises Randomized success on RoboTwin 2.0 Clean-to-Randomized transfer from 10.16% to 45.18%, a gain of 35.02 percentage points over FastWAM. Across two real-robot tasks and three OOD difficulty levels, CSWAM improves average success over FastWAM by 42.5 percentage points, from 27.5% to 70.0%.