Google DeepMind 与 UPenn 论文:可解释的 AI 文本检测器达到黑盒水平
DeepMind 用 Gemma-2-2B 做了个能说清判断理由的 AI 文本检测器,域内 F1 96.8 几乎追平黑盒,跨域还反超 17 分,做检测的都该看看。
Google DeepMind 与 UPenn 的论文用稀疏自编码器拆解 Gemma-2-2B 检测器内部活动,再由 LLM 给每个特征打标签。只保留标签可靠的特征后,域内 F1 为 96.8,接近黑盒检测器的 98.0。在未见测试数据上,基于基座模型特征的版本拿到 92.2 F1,黑盒只有 75.2。这意味着部署 AI 文本检测器时可以读取并移除其依赖的不公平线索,比如误伤学生作文和同行评审的那些。
New Google DeepMind + UPenn paper shows that an AI-text detector can explain its verdicts with readable writing features and still match a black-box detector.
suggests that anyone deploying AI-text detectors should be able to read and remove the specific cues a detector relies on.
The best AI-text detectors are black-box classifiers, yet they flag student essays and peer reviews. Readable stylometric detectors exist but trail them by wide margins.
They trained sparse autoencoders on a Gemma-2-2B detector's internals to split its activity into features, then had an LLM label each one. They kept only features whose labels predicted their behavior well and fed those into a simple classifier.
With only well-labeled features, the detector kept 96.8 F1 in-domain, close to the black box's 98.0. On unseen test data, a version built on base-model features scored 92.2 F1 against 75.2 for the black box.
If your detector's decisions affect people, build it on labeled features so reviewers can see and drop cues they consider unfair.