LeWAM: 基于扩散引导MPC的JEPA世界行动模型
LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC
LeWAM通过扩散引导MPC改进了世界行动模型，在预测和规划任务中表现优异，比传统重建模型更高效。
LeWAM是一种双向transformer模型，可在JEPA潜在空间中端到端训练，支持前向、后向、逆动力学和策略预测四种模式。研究显示，LeWAM的潜在空间能更好地读取机器人和物体状态，忽略视觉干扰因素。在闭环评估中，LeWAM与相同编码器训练的flow-matching策略性能相当，同时提供世界模型功能。
LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC
World action models (WAMs) predict actions and future observations, typically from a reconstruction-based representation that carries noisy, redundant information which can complicate downstream predictions. We introduce LeWAM, a bidirectional transformer for forward, backward, inverse dynamics and policy prediction, on a decoder-free JEPA latent trained end-to-end through all four modes. We see the following benefits: 1) Alignment: linear probes read robot and object state from LeWAM's latent better than from a regular Le World Model (a forward-only JEPA world model), while the latent ignores visual distractors as well as LeWM does and far better than a reconstruction-based WAM. 2) Acting: Closed-loop evaluations of LeWAM match a regular flow-matching policy trained on the same encoder at matched size, while also providing a world model. 3) Planning: Sampling raw actions when planning with WAMs lets MPC exploit dynamics-model inaccuracies; planning in the noise space of the policy head instead improves the closed-loop performance of these WAMs.
