AnyGroundBench:面向视觉语言模型的视频定位专用领域基准

AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models

精选理由

想看看现在的VLM在专业视频场景下有多拉胯?这篇论文搞了个AnyGroundBench基准,测了15个模型在动物、手术等5个领域的表现,结果全翻车了。

AI 摘要

AnyGroundBench是一个域适应基准测试,将时空视频定位评估从静态零样本测试转向严格的域适应。它覆盖动物、工业、运动、手术和公共安全五个专业领域,提供新拍摄视频(如专家标注的小鼠行为)与现有数据集配对,并配备密集的高保真时空标注。评估了15个SOTA视觉语言模型(VLM)的零样本泛化和上下文学习(ICL)能力。结果表明,当前模型在专业领域上的零样本和基于ICL的适应均失败,暴露出时空推理的关键缺陷。

原文 · arXiv cs.AI

AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models

Vision-Language Models (VLMs) have demonstrated immense promise in Spatio-Temporal Video Grounding (STVG). However, current evaluation protocols are largely confined to zero-shot assessments on general, daily-life benchmarks. This creates a critical disconnect from real-world applications in specialized fields, where models inevitably encounter rare visual concepts and complex spatio-temporal dynamics. Since exhaustive pre-training across infinite data distributions is infeasible, the ability to adapt to novel domains is essential. To bridge this gap, we introduce AnyGroundBench, a domain-adaptation benchmark designed to shift the STVG evaluation paradigm from static zero-shot testing to rigorous domain adaptation. Targeting five specialized domains (animal, industry, sports, surgery, and public security), AnyGroundBench pairs newly captured videos such as expert-annotated mouse behaviors with established datasets, unifying them through dense, high-fidelity spatio-temporal annotations. Crucially, the benchmark provides dedicated training subsets to systematically measure domain adaptability. We extensively evaluate 15 state-of-the-art VLMs, assessing their zero-shot generalization and In-Context Learning (ICL) capabilities under practical computational constraints. Ultimately, our findings reveal that current models fail in both zero-shot and ICL-based adaptation when confronted with specialized domains, exposing critical flaws in spatio-temporal reasoning that future research must address.