DLAM:带时间约束的分布性潜在动作模型

DLAM: Distributional Latent Actions with Temporal Constraints

精选理由

DLAM用对角高斯分布建模潜动作,解决了时间一致性差和误差累积问题,在多个机器人基准上提升了策略性能。

AI 摘要

DLAM是一种分布性潜在动作模型,将每个转移表示为对角高斯分布,通过归一化组合和反转约束均值和方差。与现有潜动作基线相比,DLAM在保留态视频上学习到更一致的时间动态,并取得更强的直接与累积重建效果。在π_0迁移协议下,DLAM在MetaWorld MT50、LIBERO和真实机器人操作任务上提升了策略性能。消融实验表明,归一化均值约束贡献了大部分重建增益,而学习方差与相关性感知组合在下游控制中提供互补改进。

原文 · arXiv cs.AI

DLAM: Distributional Latent Actions with Temporal Constraints

Vision-language-action (VLA) models remain constrained by scarce action-labeled robot data, whereas action-free videos offer abundant observations of physical change. Latent action models can extract such priors, but reconstruction-trained codes may predict future observations without the structure required for joint generation with robot actions. Existing structured methods add temporal constraints but retain deterministic transition points, so residual errors in locally inferred transitions may propagate and compound under recursive composition. We introduce DLAM, a distributional latent-action model that represents each transition as a diagonal Gaussian. Reconstruction conditioned on the reference frame grounds the mean in observed visual change, while normalized composition and reversal over equal-gap triplets constrain both the mean and dimension-wise variance. Variance composition uses a lightweight shared-correlation coefficient to account for dependence between adjacent transitions that share an intermediate frame, whereas reversal negates the mean and preserves the variance. For downstream policy learning, we freeze the encoder and train a flow-matching policy to jointly generate mean transition sequences and robot actions. On held-out transitions, DLAM learns more temporally consistent latent dynamics than existing latent-action baselines and achieves stronger direct and cumulative reconstruction on held-out videos. Under the same controlled $π_0$ transfer protocol, it also improves policy performance on MetaWorld MT50, LIBERO, and real-world manipulation tasks. Controlled ablations show that normalized mean constraints account for most of the reconstruction gain, while learned variance and correlation-aware composition provide complementary improvements in downstream control.