这篇论文提出不用大模型也能高性能做多模态指代消解，直接在 CIN 上就比专用方法和 VLLM 高 5% 和 2%，而且即插即用，挺实用的。
论文提出即插即用的多模态指代消解方法 Plug-and-Adapt，无需在目标数据集上训练即可使用。该方法先利用视觉-语言对齐数据集预训练一个细粒度对齐模型，再通过证据理论融合视觉和类别线索进行相似度聚合。在 CIN 基准测试中，CoNLL F1 比现有专用方法提升 5.31%，比主流 VLLM 提升 2.12%。在掩码 CIN 和 VCR-MCR 数据集上验证了鲁棒性和泛化能力。
Plug-and-Adapt: Multimodal Coreference Resolution at First Sight with a Pretrained Alignment Model
Visual information helps resolve ambiguity in coreference resolution, leading to notable performance gains. However, existing Multi-modal Coreference Resolution (MCR) methods require training with (partially) annotated data from the target dataset before they can be applied, preventing their direct usability and raising concerns about generalization. While Vision-Language Large Models (VLLMs) with billions of parameters offer promising zero-shot capabilities, they remain largely inaccessible. Their massive size limits deployability, and many are only accessible through paid APIs. In this paper, we propose a plug-and-adapt method that strategically adapts a carefully pre-trained \emph{alignment model} for immediate use in MCR tasks, designed to eliminate the need for training on scarce benchmark datasets or relying on resource-intensive VLLMs. Specifically, we first pre-train a fine-grained alignment model between textual and visual contextual information using vision-language alignment datasets. We then repurpose the alignment model to MCR through similarity aggregation by fusing visual and categorical cues with evidence theory, thereby enhancing effectiveness. Experiments on the Coreference Image Narratives (CIN) benchmark dataset demonstrate the effectiveness of our method, achieving a 5.31\% and 2.12\% improvement in CoNLL F1 over SOTA dedicated methods and popular VLLMs, respectively. We further evaluate our method on a masked CIN dataset for robustness testing and on a specially constructed VCR-MCR dataset for generalization assessment, with results confirming both capabilities.