TraVEL:轨迹引导的视频嵌入学习用于驾驶视频检索

TraVEL: Trajectory-Guided Video Embedding Learning for Driving-Video Retrieval

精选理由

想搞自动驾驶数据检索的可以看看这个TraVEL,用行车轨迹当训练信号,让模型更懂左转右转、加速减速,比普通微调强不少。

AI 摘要

论文提出TraVEL方法,用于提升驾驶视频检索。研究者先在nuReasoning数据集上对Qwen3-VL-Embedding进行InfoNCE微调,再引入轨迹相似度作为GRPO奖励。相比纯监督微调,TraVEL在2B模型上使纵向和横向mAP分别提升9.8和4.7个百分点,在8B模型上分别提升7.2和1.5个百分点。该方法不需要外部感知输出,推理时仅靠单向量检索。

原文 · arXiv cs.LG

TraVEL: Trajectory-Guided Video Embedding Learning for Driving-Video Retrieval

Efficiently retrieving relevant clips from large-scale driving logs is essential for data curation, model development, and safety analysis. Structured and rule-based retrieval systems can explicitly target driving events, but typically require expert-defined rules, auxiliary data, and multi-stage perception pipelines. Multimodal embedding models offer a simpler and more efficient alternative by representing each video with a single searchable vector. However, general-purpose models often rely on shortcuts from static scene context and struggle to distinguish motion-centric events, such as turning left versus right or accelerating versus decelerating. In this work, we study how to adapt a general-purpose multimodal embedding model to driving-video retrieval. We first fine-tune Qwen3-VL-Embedding on paired clips and reasoning traces from nuReasoning using an InfoNCE objective. While this stage substantially improves overall retrieval, caption supervision alone remains insufficient for fine-grained motion understanding. We therefore introduce TraVEL (Trajectory-Guided Video Embedding Learning), a motion-aware fine-tuning framework that uses ego-trajectory similarity as a reward within Group Relative Policy Optimization. Trajectories serve only as privileged training supervision; retrieval still operates on single-vector video embeddings without ego poses, expert rules, or auxiliary perception outputs. We further construct a driving-video retrieval benchmark from nuReasoning. Experiments show that TraVEL improves motion-centric retrieval across model scales: relative to SFT, it raises longitudinal and lateral mAP by 9.8 and 4.7 points at 2B, with corresponding gains of 7.2 and 1.5 points at 8B. TraVEL thus combines physically grounded supervision with efficient embedding-based search.