TGAD基准测试揭示:文本引导异常检测模型实际并未真正理解语言指令

A Structured Benchmark for Text-Guided Anomaly Detection: When Language Stops Conditioning the Decision

精选理由

做工业视觉检测的团队会发现,当前号称支持文本引导的模型其实并不听指令——TGAD基准直接戳破了这个泡沫,建议点开看看你的模型是否真的被语言控制。

AI 摘要

工业异常检测领域近年引入多模态视觉语言模型,声称可通过文本指令实现零样本或少样本检测。但现有评估方法沿用单模态基准,无法验证模型是否真正依赖文本条件。本文提出TGAD结构化基准,通过三个递进场景测试:MVTec AD上的提示敏感性测试、组件级标注扩展、以及新构建的组装面板数据集APD。测试发现,三种代表性模型(生成式大视觉语言模型、无训练判别式、嵌入自适应判别式)均仅表面响应文本指令:移除物体名词后生成模型I-AUROC从97.4降至82.6;组件指令无法约束决策;在APD上图像级判别甚至低于随机水平(最低31.5)。结果表明当前多模态异常检测系统的文本引导能力被高估,需要此类基准才能实现工业部署所需的可靠语言控制。

原文 · arXiv cs.AI

A Structured Benchmark for Text-Guided Anomaly Detection: When Language Stops Conditioning the Decision

Industrial anomaly detection has historically been a unimodal task. Recent multimodal vision-language models have produced systems that admit textual input alongside the image and are presented as enabling text-guided zero- and few-shot inspection. Yet these methods are evaluated with protocols inherited from unimodal benchmarks that hold the textual condition constant and therefore cannot measure whether language conditions the decision; whether reported gains reflect text guidance or strong pretrained visual features remains open. We introduce Text-Guided Anomaly Detection (TGAD), a structured benchmark that progressively increases the functional role of language across three scenarios: a controlled prompt-sensitivity setting on MVTec AD; a component-tagged extension of MVTec AD that requires the model to restrict its assessment to an instructed part; and the new Assembled Panel Dataset (APD), a realistic industrial setting that requires both defect-type and component-location knowledge. We evaluate one representative model per paradigm: generative large vision-language, training-free discriminative, and embedding-adaptive discriminative. In all three, the textual interface conditions the decision only superficially: prompt content is absorbed unless the object noun is removed (the generative model's I-AUROC drops from 97.4 to 82.6); component-level instructions do not constrain the decision once defects outside the instructed part are admitted as normal (from 90.3 to 66.3); and when both combine on APD, image-level discrimination collapses below the MVTec level, in one case below chance (71.2, 50.5, 31.5). These results suggest that standard benchmarks overstate the text-guided capabilities of current multimodal anomaly detection systems, and that a protocol of this kind is a prerequisite for models that can be reliably controlled through language for industrial deployment.