这个框架把时序推理和高分辨率感知结合起来,在DRAMA-Reasoning上碾压了现有方法,还能零样本泛化到NuScenes和BDD100K,代码也开源了,做自动驾驶可解释性研究的值得一看。
UniDrive是一个统一视觉语言与定位框架,旨在实现自动驾驶中的可解释风险理解。它通过时间推理分支从多帧视觉输入建模场景动态,并通过高分辨率感知分支从最新帧保留细粒度空间细节,两者经门控交叉注意力融合模块集成。基于融合表示,UniDrive联合生成自然语言风险描述和风险对象的边界框输出。在DRAMA-Reasoning基准上,UniDrive在描述和风险对象定位上均优于基于图像和视频的基线,并在小目标定位、零样本泛化至NuScenes和BDD100K方面表现突出,人类评级的可解释性和可信度也更高。
UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving
Recent multimodal large language models (MLLMs) have shown strong potential for autonomous driving scene understanding, yet existing methods still face a fundamental trade-off between temporal reasoning and spatial precision. Models that rely on single-frame or low-resolution inputs often miss small, distant, or partially occluded hazards, while language-centric driving models frequently provide limited grounded evidence for their explanations. To address this gap, we propose UniDrive, a unified visual-language and grounding framework for interpretable risk understanding in autonomous driving. UniDrive combines a temporal reasoning branch that models scene dynamics from multi-frame visual input with a high-resolution perception branch that preserves fine-grained spatial details from the latest frame. The two branches are integrated through a gated cross-attention fusion module, enabling dynamic context to be aligned with precise spatial evidence. Based on the fused representation, UniDrive jointly generates natural-language risk descriptions and grounded bounding-box outputs for risk objects. Experiments on the DRAMA-Reasoning benchmark show that UniDrive outperforms representative image-based and video-based baselines in both captioning and risk-object grounding. In particular, UniDrive achieves the best overall performance on the validation split and demonstrates clear advantages in small-object localization, zero-shot generalization to NuScenes and BDD100K, and human-rated interpretability and trustworthiness. These results suggest that explicitly combining temporal semantics and high-resolution perception provides a stronger foundation for interpretable and safety-oriented autonomous driving systems. The code is available at https://github.com/pixeli99/unidrive-dev.