ProductsMulti-source

OpenAI 发布分类器,帮助儿童性虐待举报热线筛查 AI 生成图像

OpenAI wants to stop deepfakes from overwhelming child abuse tiplines

Why we picked it

OpenAI 把自家模型改造成打击儿童性虐待图像的调查工具,能识别 AI 生成内容并帮举报热线过滤噪声。和 Thorn 等机构现有方案比,多了判断图像真伪的能力。做内容审核、信任安全工作的团队应该看看它的开放程度。

OpenAI 在华盛顿特区周一的媒体简报会上宣布推出一款分类器工具,用于帮助儿童性虐待内容调查人员从海量举报中快速识别有害图像。该分类器可检测虐待儿童图像,并判断内容是否为 AI 生成。随着生成式模型普及,举报热线收到的 AI 生成图像数量激增,人工核查负担加重。OpenAI 希望通过该工具过滤噪声,让调查人员优先处理可采取行动的真实案件材料。

Image · theverge
Body

技术机制上,该分类器由 OpenAI 的模型驱动,对提交的图像进行分类,区分真实虐待图像与 AI 生成或无害内容,从而降低核查人员逐张审阅的工作量。这一方向与 2024 年 Thorn、IWF 等机构报告的趋势一致:NCMEC 等举报渠道收到的疑似 AI 生成 CSAM 举报在近年明显上升,大量占用原本用于追查真实受害者的资源。

对行业而言,这是 OpenAI 将安全研究转化为执法协作工具的一次落地,也可能成为其他平台处理类似举报积压的参考方案。开发者与内容平台需要关注其 API 与开放程度,以及分类器在误报率上的表现如何影响一线核查流程。

Original · theverge

OpenAI wants to stop deepfakes from overwhelming child abuse tiplines

OpenAI is announcing a tool that it says will help child sexual abuse investigators cut through the noise to find harmful material they can act on quickly. At a Monday press briefing in Washington, DC, OpenAI announced a classifier it says can better detect abusive imagery of kids, including whether it's AI-generated. It's also offering […]

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI regulation, compliance frameworks, and sovereign compute policy and AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include OpenAI, AI safety, content moderation.

Source attribution: curated from theverge (@Lauren Feiner). Cross-source corroboration: 11 independent sources have covered this development (including techcrunch, Simon Willison’s Weblog, pandaily), confirming industry-wide signal strength.