Papers

JADE 让 VLM 与扩散动作专家联合去噪,提升机器人动作生成

Vision-Language Models Are Joint Denoisers in Robotic Action Generation

Why we picked it

机器人领域的新论文,JADE 让 VLM 直接参与每一步动作去噪,四个真实双臂任务比 π0.5 高 20.8 个百分点,做 VLA 的可以看看。

论文提出 JADE,让 VLM 在每个去噪步骤中读取带噪动作、视觉观测和语言指令,通过 gated cross-attention 引导动作专家联合去噪。VLM 还被监督从带噪动作潜变量中恢复干净潜变量,获得直接的动作学习信号。在 RoboDojo 基准上,JADE 在 π0.5 系方法中平均成功率达 12.67%,为最高。在 RoboTwin 2.0 上也有提升,四个真实世界双臂任务的平均成功率比 π0.5 提高 20.8 个百分点。

Original · arXiv cs.AI

Vision-Language Models Are Joint Denoisers in Robotic Action Generation

Diffusion-based vision-language-action (VLA) models cascade pretrained vision-language models (VLMs) with diffusion action experts, but typically restrict the VLM to providing static conditioning. This separation limits how much pretrained perception and reasoning can guide action generation, especially in out-of-distribution settings. We introduce JADE, a Joint Action DEnoiser that enables the VLM and action expert to jointly refine actions. Our key insight is that structured action latents allow the VLM to read the current noisy action together with visual observations and language instructions. At every denoising step, the VLM contextualizes the noisy action and guides the action expert through gated cross-attention, so that its guidance evolves with the action being generated. We further supervise the VLM to recover clean action latents from noisy ones, which gives it a direct action-learning signal and couples perception with control. Additionally, joint denoising enables JADE to generate actions with fewer sampling steps, helping offset the additional cost of VLM participation. An adaptive step-size schedule further balances task performance and inference latency. On RoboDojo, JADE achieves an average success rate of 12.67%, the highest among the evaluated $π_{0.5}$-based methods, and is competitive with recent VLA and WAM foundation models pretrained at scale. It also improves performance on RoboTwin 2.0 and raises the average success rate across four real-world bimanual tasks by 20.8 percentage points compared with $π_{0.5}$.