这篇论文搞了个新度量TPIPS，能按文字指令判断两张图在哪个方面像，比现有单标量打分更细。数据集和模型都开源了。
本文提出TPIPS，一种基于文本提示的图像感知相似度度量，能根据文本条件区分形状、颜色等不同视觉相似性维度。研究基于大规模三元组数据集，包含多种自由形式语义相似性标注。在多个前沿视觉语言模型（VLM）基准测试中，模型与人类一致性差距显著。通过微调VLM，TPIPS在文本引导检索、组合搜索和生成模型细粒度评估中表现更优，且泛化良好。
The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric
Human visual similarity judgments are context-dependent. For example, two images may be similar in shape but distinct in color. Existing perceptual similarity metrics, however, collapse these nuances into a single scalar value, offering no mechanism to condition on specific aspects. To bridge this gap, we introduce a large-scale dataset of human similarity judgments over image triplets, where each triplet is annotated across multiple, free-form semantic aspects of similarity. Benchmarking a broad range of frontier vision-language models (VLMs) reveals a considerable performance gap compared to human annotators' consensus. Leveraging our data, we fine-tune a VLM to produce our Text-Prompted Image Perceptual Similarity (TPIPS) metric, capturing multiple senses of visual similarity depending on the specified text prompt. We demonstrate that TPIPS aligns more closely with human perception and generalizes reliably beyond the training distribution. Finally, we show that TPIPS unlocks new capabilities in text-guided retrieval, compositional search, and the fine-grained evaluation of generative models. Our code, data, and trained models are at https://peterwang512.github.io/TPIPS