这项研究戳破了AI文本检测的底层假设——检测器可能只是在放大预训练模型的偏见而非真正学习区分。做AI安全、内容审核或学术诚信检测的团队,看完会重新评估现有检测方案的有效性。
最新研究发现,AI文本检测器并非真正学习区分人类与AI写作,而是放大预训练模型中的典型性方向。在RoBERTa-base等架构上,直接投影质心差异即可达到甚至超过微调后的检测性能(AUROC 0.806-0.944)。该方向在非母语英语写作上完全反转(AUROC低至0.06),验证了典型性假说。仅需24个样本的冻结探针即可匹配全微调效果(0.900 vs 0.895)。研究还提出闭式雅可比预测器,可精确操控检测方向,将ELECTRA检测器在1%假阳性率下的真阳性率从0提升至0.904。
Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction
AI text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary. On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures (86-106% of the fine-tuned discrimination ceiling: on RoBERTa-base, raw projection exceeds fine-tuning); on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested. The same axis inverts on non-native ESL writing (AUROC 0.06-0.20) -- a falsifiable prediction unique to the typicality reading. A 24-example frozen probe matches full fine-tuning (0.900 vs 0.895). A closed-form Jacobian predictor parameterises axis-manipulating interventions with R^2 = 1.000 universal, lifts ELECTRA-CE deployment TPR from 0.000 to 0.904 at FPR = 1%, and transfers to three independently-trained third-party RoBERTa detectors at 16/16 oracle-equivalence (57% NYT-FPR reduction on the OpenAI detector). Scope: encoder family; mechanism magnitude HC3-anchored; population-level shared axis with per-text mechanisms varying across architectures. Three operationally distinct probes -- text-surface caps_rate residualisation, geometric signed-epsilon ablation, closed-form text-pair predictor -- agree at cos 0.74/0.81/1.00 across three architectures, confirming observer-invariance. Under matched-TPR-0.90 evaluation, the published intervention zoo (CC, dealign-f2c) is calibration-equivalent across 27 cells (|Delta AUROC| <= 0.0081), and >= 97% of the LoRA->full-FT bias gap on ELECTRA is calibration shift, not learned representation -- the central claim's prediction confirmed.