OpenAI 联手 Goodfire 检视模型内部,思维链可读性正在下降
OpenAI 发现自家模型的思维链越来越难读懂了,于是找做可解释性的 Goodfire 进模型内部查原因。搞推理模型监控或安全审计的人该关注,这说明光看思维链已经不够可靠了。
OpenAI 正在与模型可解释性公司 Goodfire 合作,检查其模型内部机制。合作的核心动因是:OpenAI 的研究员发现模型的思维链(chains of thought)质量正在退化——其中包含的信息变少,且越来越难以被人读懂。
这条解释来自 rocketalignment,通过 The Information 的推文发布。问题的实质在于,如果思维链不再忠实反映模型的真实推理过程,开发者和安全团队就无法通过阅读推理过程来判断模型为何给出某个答案。Goodfire 的可解释性工作正是为了绕开表层文本,直接探查模型内部状态。
OpenAI 与 Goodfire 的合作意味着可解释性正从前沿实验室的内部研究走向外部采购,Anthropic 等公司此前也长期投入该方向。对于依赖思维链做监控的企业用户来说,推理过程失真直接影响对模型的审计能力。
.@rocketalignment explains why OpenAI is working with Goodfire to examine what happens inside its models:
"The trouble is that chains of thought seem to be degrading. They seem to have less information in them. They're less and less legible." https://t.co/vEQoeHbI48
This report is categorized under AITOP's "Industry" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include OpenAI, Goodfire, chain-of-thought, interpretability.
Source attribution: curated from The Information (@theinformation).