这篇论文研究了AI模型评估结论过时的速度,发现模型发布后近一年结论可能已失效。如果你做AI评测或读评测报告,它能帮你判断证据是否还新鲜。
这篇论文审计了2025年7月18日至2026年7月17日期间40条实证记录,发现最新命名模型出现时中位年龄为281天(四分位距75-478天,范围11-939天)。其中25篇期刊文章的中位年龄为395天,14篇预印本为56天,1篇实验室报告为49天。35条记录包含已被取代的模型系列,仅7条提供了精确的日期标识符。论文将模型年龄与声明货币性区分开,提出六项报告实践,并展示了作者借助GPT-5.6 Sol Pro在两天内完成研究的反思性案例。
The Half-Lives of Generative-AI Evidence: A 40-Record Audit, a Claim-Currency Framework, and a Reflexive Case of Frontier-Model-Assisted Research
Generative-AI evaluations can become historical before publication, yet calendar age does not affect every conclusion equally. This paper has two linked purposes. First, it audits a maximum-variation purposive corpus of 40 empirical records appearing between 18 July 2025 and 17 July 2026. The audit coded publication route, execution timing, model identity, age of the newest named generation or immutable snapshot, same-family supersession and refresh behaviour. At appearance, the newest named model was a median 281 days old (middle 50%: 75-478; range: 11-939). Median age was 395 days for 25 journal articles, 56 days for 14 preprints and 49 days for one laboratory report. Thirty-five records included a superseded family, seven supplied a precise dated identifier, three clearly refreshed model evidence, and one added a late sensitivity test. All 40 included an OpenAI system, a feature of this corpus rather than a prevalence estimate. The paper distinguishes model age from claim currency and proposes six reporting practices. Second, it treats its own two-day production process as a reflexive case of frontier-model-assisted research creation. GPT-5.6 Sol Pro in ChatGPT supported candidate discovery, source reconciliation, calculations, drafting and critique; the author checked sources, made all substantive decisions and accepts responsibility. This is a proof-of-practice, not a controlled estimate of productivity or quality. By applying its own Model Facts and model-currency statement, the paper shows how rapid AI-assisted research can be made inspectable without treating model output as independent validation. The title uses half-lives metaphorically; no universal decay rate is estimated.
- Gary Marcus07-25 18:41原文