做模型可解释性研究的团队会感兴趣——这篇论文把共激活聚类和因果消融结合起来，给出了一个验证注意力头电路的实用方法，建议做电路分析的开发者试试这个闭环流程。
这篇论文提出了一种通过共激活统计聚类注意力头、再用因果消融验证电路的方法。在Pythia 1B和OLMo 1B两个密集模型上，聚类发现的社区通过了消融测试，表明共激活信号能有效识别电路。但在混合专家模型OLMoE-1B-7B中，路由条件聚类虽能恢复统计信号，但消融后反而改善损失，说明信号无效。作者强调，共激活只是电路提案，消融才是确认电路的关键。研究揭示了注意力头选择性和参与度在训练过程中与功能解耦的现象。
Closure-Validated Circuit Discovery in Attention Heads: Co-activation Proposes, Ablation Disposes
Interpretability increasingly treats groups of components, not individual units, as the basic object, and proposes to find them by clustering co-activation statistics. We ask whether such a cheap signal actually identifies an attention-head circuit. Adapting a sparse-autoencoder clustering recipe to attention heads -- but validating by causal ablation rather than reconstruction -- we cluster heads and then run a closure test: ablate the discovered community and compare per-example damage to matched-random controls. Across two dense 1B-scale models (Pythia 1B, OLMo 1B) and two input distributions, the communities pass closure. In a Mixture-of-Experts model (OLMoE-1B-7B), route-conditional clustering recovers a statistically real signal that nonetheless does not survive closure -- ablation improves loss, the wrong direction. Extending closure across training, attention-target selectivity and participation ratio decouple from function in both directions. We conclude that a cheap signal is a circuit proposal, not a confirmed circuit; closure is what separates them.