OpenAI 公布内部困惑度数据,提议将其作为跨模型比较的共同指标
训练损失的数字门槛被 OpenAI 摊开讲了:0.8 以下解锁网络安全和数学能力,0.68-0.7 可能带来下一个能力跃迁。做模型评测或安全治理的人都该看看这套度量提议,它可能成为榜单之外的另一种行业标准。
OpenAI 首次公布其内部困惑度(perplexity)测量数据,并提议将困惑度作为各模型开发者之间共享的比较指标。数据显示 GPT-3.5 的 log-perplexity 约为 0.99,GPT-4 降至约 0.91。
OpenAI 在文中给出了具体的能力阈值:当损失(loss)降到 0.8 以下时,模型在网络安全和数学领域能力明显增强;而约 0.68-0.7 的区间被认为是带来进一步变革性能力的窗口。这些数字首次让外界可以用同一把尺子衡量不同代际模型的内部进展。
OpenAI 还提出了一套面向开发者的共同上限方案,在 AI 监控技术达到具体里程碑之前持续生效。如果其他模型开发者(如 Anthropic、Google)接受这一度量框架,行业将出现一个不依赖基准测试榜单的、基于内部训练指标的能力比较方式。
OpenAI published its internal perplexity measurements and proposes perplexity as a shared metric to compare model developers
GPT-3.5 was at a log-perplexity of ~0.99, GPT-4 at ~0.91, and losses below 0.8 brought capabilities in cybersecurity and mathematics
Further transformative capabilities are expected in the ~0.68-0.7 window
Proposed common limits for developers would hold until AI monitoring reaches a concrete milestone
https://t.co/0ig0hiTFcs
This report is categorized under AITOP's "Industry" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include OpenAI, GPT-4, AI safety, reasoning models.
Source attribution: curated from Tibor Blaho (@btibor91). Cross-source corroboration: 11 independent sources have covered this development (including rohanpaul_ai, The Information, Bindu Reddy), confirming industry-wide signal strength.