做视频理解或 MLLM 研究的同学,这篇综述帮你把碎片化的方法统一到“观看-记忆-推理”框架下,省去自己梳理文献的时间,值得收藏作为 roadmap。
这篇综述从人类视角出发,系统梳理了多模态大语言模型(MLLM)在视频理解中的三大核心能力:观看(感知)、记忆(上下文保持)和推理(生成可靠输出)。文章提出统一框架,将视频理解系统分解为感知表征、记忆状态、推理轨迹和最终预测,并分析了时空感知、长视频高效处理、流式理解、忠实推理等关键挑战。作者按功能分类介绍了代表性方法,涵盖细粒度感知、多模态对齐、离线/流式记忆、纯文本与视频推理等方向,并讨论了第一人称、体育、教学、医疗等应用场景及评估基准。最后指出了可扩展、记忆感知、证据驱动的视频智能的未来方向。
Watch, Remember, Reason: Human-View Video Understanding with MLLMs
Video understanding is being rapidly transformed by multimodal large language models (MLLMs), as research moves from short clips to long, multimodal, and knowledge-intensive video scenarios. These scenarios require models to handle sparse evidence, long-range dependencies, multimodal alignment, and reliable inference under limited computational budgets. This work presents a human-view perspective on LLM-based video understanding, organized around three functional abilities: watching, remembering, and reasoning. Rather than treating video tasks as isolated benchmarks, this view provides a unified structure for analyzing how video MLLMs acquire evidence, preserve context, and produce grounded outputs. We introduce a formulation that characterizes video understanding systems by their perceptual representations, memory states, reasoning traces, and final predictions. Based on this formulation, we identify challenges in spatio-temporal perception, efficient long-video processing, memory modeling, streaming understanding, and faithful reasoning. Representative methods are organized by their roles in video MLLM systems. Watching covers fine-grained, comprehensive, audio-visual, and efficient perception. Remembering includes offline and streaming memory, while reasoning covers text-only reasoning and thinking with videos. We further examine application domains such as egocentric, sports, instructional, medical, and narrative videos, and cover training datasets and evaluation benchmarks across task types, supervision formats, modalities, and capability dimensions. Finally, we outline open problems and future directions for scalable, memory-aware, and evidence-grounded video intelligence. Related works will be continuously traced at https://github.com/marinero4972/Awesome-HumanView-VideoUnderstanding.