想测你手上的AI图片检测器靠不靠谱?这个基准有8602张图、6种类型,还试了5种检测器,发现它们碰上有文字的图片(比如收据、海报)就露馅,连JPEG压缩都扛不住。
该论文提出了一个多领域基准数据集,包含8602张图像,覆盖商业海报、信息图、学术海报、收据、表格和UI截图6个类别,用于检测OpenAI的GPT Image 2生成的文本丰富图像。在零样本设置下评估了5种AI生成图像检测器,发现性能高度依赖领域:在部分类别表现好的方法在其他类别失败,且最强传统检测器对JPEG压缩敏感。此外,多模态视觉语言模型的初步评估显示其在结构化格式上既有潜力也有局限。
A Multi-Domain Benchmark for Detecting AI-Generated Text-Rich Images from GPT-Image-2
Text-rich images often contain privacy-sensitive, transactional, or decision-relevant information. As recent multimodal image generation models become increasingly capable of synthesizing realistic textual content and structured visual designs, detecting AI-generated text-rich images has become an important challenge for digital trust and content authenticity. Existing benchmarks, however, largely focus on object-centric images and provide limited coverage of scenarios where textual semantics and layout organization are central. In this paper, we introduce a multi-domain benchmark for detecting text-rich images generated by OpenAI's GPT Image 2. The benchmark contains 8,602 images across six representative categories: commercial posters, infographics, academic posters, receipts, tables, and UI screenshots. Using this benchmark, we evaluate five representative AI-generated image detectors in a zero-shot setting and analyze their overall, category-wise, and post-processing robustness. Our results show that detector performance is highly domain-dependent: methods that perform well in some categories often fail on others, and even the strongest conventional detector exhibits severe sensitivity to JPEG compression. We further conduct an exploratory evaluation with a multimodal vision-language model, revealing both its promise and its limitations on structured formats. These findings highlight the need for text- and layout-aware detection methods for modern AI-generated images. Our dataset is released at XXX.