Nick Cammarata 新文：神经元可解释性研究曾被误读，Qwen3-32B 审计显示 90% 神经元有明确职能
Zoom In 论文作者自己出来纠正旧结论，Qwen3-32B 上 90% 神经元有明确职责，做可解释性的人应该看看这篇。
Nick Cammarata 作为 2020 年 Zoom In 论文共同作者，在 Goodfire 发布的文章中称学界此前误读了多义神经元的关键证据。他对 Qwen3-32B 的初步审计显示约 90% 的神经元信号有单一主要职能，仅 2-4% 尚未理解，而他 2023 年在 OpenAI 启动的自动化研究当时只判定约 1.7% 可读。文中还引用 GPT-6 Astra 的 System Card 指出其推理比前代更难监控，并提到下一代模型可能采用更多以数字而非可读文本思考的 Loop Transformers。不过该审计基于经典 Transformer 架构测试，距离实时监控应用尚有距离。
Very intersting read: Nick Cammarata, a co-author of the 2020 Zoom In paper that introduced the idea that neural networks fold unrelated concepts into the same neurons, now says the field misread its own key evidence. In an essay published Thursday by interpretability startup Goodfire, he reports that in a preliminary audit of Qwen3-32B, roughly 90% of neuron signals have one main job and only 2-4% are not yet understood; the automated study he started at OpenAI in 2023 judged about 1.7% legible with the tools of the time.
After all, let’s remember: According to OpenAI’s own System Card, GPT-6 Astra's reasoning is harder to monitor than that of its predecessors. According to The Information, the next model is expected to rely on Loop Transformers, which think more in numbers and less in readable text.
That sounds like a solution: if the text disappears, you just look inside. However, this is still far from being a viable real-time monitoring method, and the testing was conducted on classic transformers, not on loop architectures. Still, it's a good step in the right direction and can be a great help in understanding the system, and thus in ensuring its safety.
