这篇论文用逆运动学原理解决了Driving VLA忽视视觉特征的顽疾,做自动驾驶轨迹规划的团队值得关注——0.5B模型就能达到7B-8B的效果,意味着更低的部署成本和更好的视觉鲁棒性。
现有驾驶视觉-语言-动作模型(Driving VLA)在轨迹预测时严重忽视视觉特征,原因是任务定义存在结构性缺陷。研究者从逆运动学角度重新设计,要求模型预测未来视觉状态作为边界条件,并引入独立逆运动学网络(交叉注意力条件扩散模型)来抑制对自车状态和文本指令的捷径依赖。仅用0.5B参数,该模型在NAVSIM-v2和nuScenes基准测试中达到7B-8B大模型的轨迹规划性能,尤其在动态驾驶场景(如转弯)中视觉特征利用显著提升。
Grounding Driving VLA via Inverse Kinematics
Existing Driving VLAs predict trajectories while largely ignoring their visual tokens -- a phenomenon we trace not to insufficient training but to a structurally ill-posed task formulation. We show that trajectory recovery, when viewed through the lens of inverse kinematics, requires both a current and a future visual state as boundary conditions; existing VLAs supply only the former, which encourages the model to shortcut through ego status and text commands alone. To address this, we re-design Driving VLA in the style of an inverse kinematics solver. First, a next visual state prediction objective that requires the LLM to predict the future visual scene provides dense visual supervision and suppresses shortcut paths. Second, a separate Inverse Kinematics Network (a cross-attention-based conditional diffusion model) that takes only the current and future visual states as input is designed to suppress reliance on ego status and textual shortcuts during trajectory decoding. With this simple prescription alone, our 0.5B-scale model recovers visual grounding and reaches trajectory planning performance comparable to 7B--8B VLAs more than an order of magnitude larger, on both the closed-loop NAVSIM-v2 and the nuScenes benchmarks. Extensive analysis further shows that this improvement stems from a recovered ability to exploit visual features, with the effect being most pronounced in dynamic driving situations such as turning.