EditSafetyBench 揭示图像编辑系统存在文本与视觉安全脱节
Text-Aware, Vision-Blind: How Multimodal Image Editing Systems Handle Unsafe Inputs
如果你在做图像生成或编辑类产品,这篇论文值得认真看:它用 5,940 个测试对证明 Gemini、GPT、Qwen 都会漏掉带风险的图片,只要指令写得中性就照样执行。安全审核只盯提示词的做法在这篇面前站不住脚。
一项新研究发布 EditSafetyBench 基准,用于衡量指令式图像编辑系统的系统级安全能力,包含 5,940 个图像-指令对,覆盖 11 类风险和三种编辑意图。
研究测试了 Gemini、GPT、Qwen API 和 Qwen Local 四个系统,发现明显的文本-视觉安全错位:Gemini 和 GPT 对含不安全意图提示词的拒绝率为 87.1% 和 81.6%,但面对不安全图像配中性指令时拒绝率骤降至 24.2% 和 22.4%,说明视觉侧风险检查形同虚设。
风险类别层面各系统标准也不一致,例如对不安全政治内容,GPT 放行 70.4% 的编辑请求,Gemini 只放行 34.8%。性内容是唯一视觉敏感度最高的类别,作者认为得益于成熟的 NSFW 识别技术。结论指向图像编辑系统需要同时推理视觉与文本两个模态的安全护栏。
Text-Aware, Vision-Blind: How Multimodal Image Editing Systems Handle Unsafe Inputs
Multimodal image generation systems have made rapid progress and increasingly emphasize content safety. However, their safety mechanisms tend to rely on a single modality and lack joint consideration of both visual content and text prompts, particularly in instruction-guided image editing, where the uploaded image may itself carry risk. We thus introduce EditSafetyBench, a benchmark that measures system-level safety awareness of advanced image editing systems, containing 5,940 image-instruction pairs, covering safe and unsafe images across 11 risk categories and three edit intents. We then probe the benchmark dataset on four systems, including Gemini, GPT, Qwen API, and Qwen Local. Based on our evaluation, we observe a systematic text-vision safety mismatch: Gemini and GPT refuse 87.1\% and 81.6\% cases containing unsafe-intent editing prompts, while only refusing 24.2\% and 22.4\% cases with both unsafe images and neutral instructions. At the category level, most risks are considered inconsistently by different systems, e.g., GPT allows 70.4\% editing attempts of unsafe political content while Gemini only allows 34.8\%. Notably, sexual content shows the strongest visual sensitivity, likely benefiting from mature NSFW identification works. Our findings highlight the need for safety guardrails that jointly reason over both visual and textual modalities in image editing systems.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include EditSafetyBench, Gemini, GPT, Qwen.
Source attribution: curated from arXiv cs.LG (@Ziqing Yang, Yixin Wu, Michael Backes, Yang Zhang).