VLM先看后合成，让视频字幕生成更精准，比传统方法更智能地找到事件变化点。
研究人员提出Seeing Before Synthesizing (SBS)框架，解决弱监督密集视频字幕生成中的视觉对齐问题。该方法利用视觉语言模型(VLM)为事件间间隙生成帧级叙事，并检测语义变化点。在ActivityNet Captions和YouCook2数据集上，该方法在字幕生成和事件定位任务上达到最先进性能。
Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning
Weakly-Supervised Dense Video Captioning aims to localize and describe multiple events in untrimmed videos given only an ordered set of event-level captions per video. Recent work synthesizes auxiliary transition captions via LLM to provide additional vision-language alignment, but these captions lack visual grounding and are rigidly assigned to every inter-event gap at a fixed location and duration. To address these, we propose Seeing Before Synthesizing (SBS), a framework that adaptively provides visually grounded linguistic guidance only where warranted. Leveraging a VLM, we generate frame-level narratives for the inter-event gaps and detect transitions from the semantic variation across them. For identified transitions, we then refine inter-event temporal masks by blending the temporal midpoint with the semantic change point and selecting the width that maximizes vision-language alignment. Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art performance in both captioning and localization.