Papers

LoCaR:基于冻结视觉语言模型的局部画布检索框架

LoCaR: Localized Canvas Retrieval With Frozen Vision-Language Model

Why we picked it

用 SAM2 加冻结视觉编码器做免训练的局部检索,专门解决整图嵌入看不清小物体的老毛病,还附赠一个 LOCFG 基准可以直接跑。做图像搜索或 RAG 检索的同学可以照着复现,不用改任何底座模型权重。

arXiv 论文提出 LoCaR,一个免训练的局部文本到图像检索框架,使用冻结的视觉语言模型。它借助 SAM2 生成的独立候选区域,对每个区域单独编码,解决整图嵌入难以表征小物体和局部属性的问题。

Body

框架还引入配对支撑适配作为可选残差模块,利用匹配的文本区域样本进行微调而不更新底层编码器。论文同时发布 LOCFG,一个从 Visual Genome 派生的基准,专门评估针对更小、更具挑战性指称对象的局部检索。

跨多个视觉语言骨干网络的实验显示,提升区域表征本身是性能提升的主要来源。该方法对所有需要精确检索局部内容的下游系统(如图像搜索、智能体视觉工具)都可直接套用,且无需训练成本。

Original · arXiv cs.AI

LoCaR: Localized Canvas Retrieval With Frozen Vision-Language Model

Whole image embeddings can under represent small objects and local attributes that distinguish relevant images for localized queries. We present LoCaR, a training free framework for localized text to image retrieval with frozen vision language models. LoCaR constructs and utilizes independent proposals (SAM2) focusing on the object so as to get a more localized representation that independently encodes each region using the frozen visual encoder. We further study paired-support adaptation as an optional residual that uses matched text region exemplars without updating the underlying encoders. We also introduce LOCFG, a Visual Genome derived benchmark designed to evaluate localized retrieval for smaller and more challenging referents. Across several vision language backbones, our results show that improving the region representation itself is the main source of gain.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LoCaR, SAM2, Visual Genome, multimodal.

Source attribution: curated from arXiv cs.AI (@Rahim Hossain, Dipanwita Saha, Ashirbad Mishra, Xinxin Shu, Svetlana Strunjas, Binbin Li).