LAG-Fusion:异步多模态扩散策略的延迟感知指导融合框架

Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion

精选理由

一篇关于多模态机器人模仿学习的论文,提出LAG-Fusion解决不同传感器速率不同的异步融合问题,实验证明比同步融合更高效。

AI 摘要

LAG-Fusion提出一种延迟感知指导融合框架,用于异步多模态扩散策略组合。该框架允许不同模态的策略以各自原生推理速率运行,并在可用时贡献去噪指导。通过推导扩散变量在相对动作表征下的参考帧重基规则,实现延迟指导在融合前的对齐。在接触丰富操作任务中,低频视觉策略与高频力觉策略的异质延迟实验显示,LAG-Fusion相比同步融合和专门设计的力感知基线,提升了策略响应速度和任务性能。

原文 · arXiv cs.AI

Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion

Diffusion policies have shown strong potential for robotic imitation learning, and recent extensions incorporate additional modalities to improve manipulation performance. However, these modalities often differ not only in information content but also in sensing rates and inference latencies. Existing multimodal diffusion policies typically rely on synchronous fusion or manually designed multi-frequency architectures, which either slow down high-frequency feedback or limit extensibility to new modality combinations. We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition. LAG-Fusion allows modality-specific policies to operate at their native inference rates and contribute denoising guidance whenever available. To make asynchronous composition consistent, we derive a reference-frame rebasing rule for diffusion variables under relative action representations, enabling delayed guidance to be aligned before fusion. We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy. Experiments under heterogeneous modality latencies show that LAG-Fusion improves policy responsiveness and task performance over synchronous fusion and specially designed force-aware baselines.