研究：短文本 AI 检测器误报率随长度骤变，提出分长度校准方法
How Short Is Too Short? Length-Conditional Calibration and Abstention for Detecting AI-Generated Text
做内容审核或学术诚信检查的会想看这篇：它用公式证明检测器在 10 token 短文本上误报率会从 1% 飙到 29%，Binoculars 真阳性率最高也只有 59%，还给了分长度校准和自动弃权的具体做法，可以直接照着实现。
一篇 arXiv 论文（arXiv:2611.10901）研究 AI 生成文本检测器在短文本上的失效问题。作者给出闭式公式，描述长文本上设定的阈值在文本变短时误报率如何变化，以及检测器达到目标真阳性率所需的最少 token 数。论文用 conformal p 值按长度分桶校准阈值，并加入分桶弃权规则：检测器在该长度下不具信息量时直接拒绝判定。
在三个零样本检测器（log-likelihood、log-rank、Binoculars）、三个语料库和两个开源权重生成模型上的实验显示，分桶校准在 10 到 200 token 的每个检测器-长度单元格中把平均误报率稳定在 5.0%，基于 300 次随机文档级重划分取均值。相比之下，单一在 200 token 设定的阈值在 10 token 处让 Binoculars 误报率达 29%，log-likelihood 约 1%，与闭式公式预测一致。
论文还揭示检测能力的天花板：没有任何检测器在短文本上达到 80% 真阳性率，最好的 Binoculars 在 200 token 处也只有 59%。因此按 80% 底线时规则在每个单元格都弃权；底线降到 0.3 和 0.5 时，才只放出 200 token 的分桶结果。
How Short Is Too Short? Length-Conditional Calibration and Abstention for Detecting AI-Generated Text
Detectors of AI-generated text are usually tuned and tested on passages of several hundred tokens, yet many real cases are short. We study score-based detection as a function of passage length. We give a closed form for how the false-positive rate of a threshold set on long text changes as length falls, and a formula for the number of tokens a detector needs to reach a target true-positive rate. We calibrate thresholds within length bins with conformal p-values, and we add a bin-level rule that abstains when the detector cannot be informative at that length. On synthetic scores the formulas agree with simulation, and binned calibration holds the false-positive rate at its nominal value at every length. On three zero-shot detectors (log-likelihood, log-rank, Binoculars), three corpora and two open-weight generators, binned calibration gives a mean false-positive rate of 5.0% in every detector-length cell from 10 to 200 tokens, averaged over 300 random document-level resplits. A single threshold set at 200 tokens gives 29% for Binoculars at 10 tokens and about 1% for log-likelihood, as the closed form predicts. No detector reaches an 80% true-positive rate, and the best, Binoculars at 200 tokens, reaches 59%, so the rule abstains in every cell at that floor and releases only the 200-token bins at floors of 0.3 and 0.5.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Binoculars, AI生成文本检测, conformal prediction, 误报率.
Source attribution: curated from arXiv cs.AI (@Ritesh Janga, Justin Zhan).