Elorian 发布 Glimpse:让模型在视频推理中回看并裁剪画面
Elorian 做了个叫 Glimpse 的视频理解模型,看视频时会回放、裁剪、抽帧,甚至跑代码对比画面变化。
Elorian 推出的 Glimpse 专注于视频时序理解,通过对比不同时间点的场景变化来理解视频。当变化幅度小或速度快时,模型会回到源视频进行更细致的查看。Glimpse 在推理过程中可以裁剪特定区域、抽取指定视频帧,或运行代码来度量所见内容。Elorian 把这套机制称为 visual scratch space(视觉草稿空间),并把这种基于帧的推理方式称作 visual thinking。
Understanding a video means comparing moments, and models are getting much better at it.
Glimpse from @ElorianAI checks how a scene changes from one point in time to another, and goes back for a closer look when the change is small or fast.
Elorian calls this visual thinking: reasoning over what the model observes, grounded in the frames.
Glimpse can return to the source during reasoning, crop into an area, extract specific video frames or run code to measure or compare what it sees, a mechanism Elorian calls a “visual scratch space.”