The Information 的独家爆料,细节很鲜:OpenAI 想买可解释性公司 Goodfire,Altman 点名欣赏前员工 Cammarata 的文章。想了解大厂怎么抢可解释性人才、这领域为何吃香的话,这篇值得看看。
#mechanistic interpretability
如果你做过浏览器操作智能体就会懂:它老点不存在的按钮。这篇用 Qwen2.5-7B-Instruct 做了很扎实的机制分析,探针预测 0.80 AUC、steering 纠 85% 幻觉,但格式一变就失效,把可解释性方法的局限讲得很诚实。做智能体评测和可解释性研究的人都该看看。
数学家和 AI 研究者合训了个小 Transformer,让它学三角形铺砌对称群的语言,然后靠看模型注意力模式想出并证明了一个数学猜想。想做可解释性或数学交叉研究的人可以看看这个范式。
Anthropic 在研究直接改模型内部表征,让它以为自己没被测试,有点像给 AI 用药改变信念,安全评估思路挺新。
The survey's value lies in its proposed evaluation framework and its six-way taxonomy of research orientations, which gives structure to a field that has grown mostly through scattered techniques. Researchers and safety teams can use the released notebooks to reproduce methods hands-on rather than reading only about them. The comparison between mechanistic evidence and behavioural evaluation is a useful lens for anyone deciding whether to invest in interpretability tools over standard safety testing.
有位研究者聊了聊她读 Anthropic 机制可解释性论文的真实感受,还吐槽了里面拟人化的 belief 写法,观点挺直接。
有人用机制可解释性把模型里的“隐喻开关”找到了,能让输出变比喻或变直白,研究文本生成风格的可以看看方法细节。
The argument pushes back on a popular narrative in mechanistic interpretability rather than adding new data, reminding readers that monosemanticity assumptions may not survive contact with real networks. Researchers and safety practitioners evaluating SAE-style interpretability pipelines should weigh this critique when deciding how much to trust neuron-level explanations. No regional angle applies; the debate is methodological.
Goodfire 的首席科学家在播客里说 AI 可能把可解释性研究加速十倍,他还讲了怎么让 tokens 产出连贯研究,做安全方向的朋友可以听听。
The notable shift is methodological: scale plus automated agents, not new theory, appears to have cracked neuron legibility, echoing the bitter-lesson pattern from other AI fields. Interpretability researchers and alignment teams should test these automated pipelines on their own models rather than assuming internals remain opaque. The claims come from a practitioner essay rather than a peer-reviewed paper, so treat the quantitative picture as one lab's report.
The method reduces AI model complexity while preserving functionality, making neural networks more interpretable and efficient for researchers and developers.
这篇论文用几何视角解释LLM评判偏见,发现偏见隐藏状态的低维子空间,还能预测模型在未知基准上的失败,挺有意思。
这篇论文提出了CircuitLasso,能以更低成本达到和现有方法一样好的电路学习效果,还能揭示可解释的语义特征如何传播。