这篇论文搞了个新任务ICQ,让AI对着人像照片在视频里找人、看行为,还放出1377个视频测试集和75K训练集,开源模型里ISYV-Model效果最好。
该论文提出ICQ任务,要求模型同时关联输入视频和人物参考图像,完成身份定位、行为理解与时间推理。配套的ISYV-Bench包含1,377个真实复杂视频和1,377个问答对,分为6个难度等级。训练集ISYV-75K提供75K个高质量样本,经自动标注和人工审核构建。实验显示主流闭源和开源多模态模型在ISYV-Bench上表现吃力,尤其在跨域身份匹配和长时跟踪方面;ISYV-Model超过强基线,部分指标接近闭源模型。
I Seek You in Videos: Identity-Conditioned Queries for Person-Centric Video Reasoning
Real-world video reasoning often involves multimodal, multi-source inputs, whereas existing video reasoning tasks typically assume a simplified video-text setting, limiting identity matching and person-centric reasoning. To bridge this gap, we introduce the Identity-conditioned Queries (ICQ) task, in which models are required to jointly associate and interpret an input video and a reference image of a person, and leverage this conditioning to address identity grounding, behavior understanding, and temporal reasoning, among other challenges. Building on ICQ, we present ISYV (I Seek You in Videos), a systematic solution comprising three components: (1) ISYV-Bench, a challenging evaluation benchmark with 1,377 real-world complex videos and 1,377 question-answer pairs, organized into six difficulty levels spanning capabilities from identity recognition to causal reasoning; (2) ISYV-75K, a large-scale training set of 75K high-quality samples constructed via automated annotation, multi-stage verification, and manual review; and (3) ISYV-Framework, containing an ICQ-oriented model and training strategy for learning to exploit informative video shots without additional shot-level annotations. Extensive experiments show that both mainstream closed-source and open-source MLLMs struggle on ISYV-Bench, especially in cross-domain identity matching and long-horizon tracking. ISYV-Model outperforms strong baselines and in some aspects approaches closed-source performance. Overall, ISYV provides a unified task definition, scalable datasets/benchmarks, and modeling insights for person-centric video reasoning.