这篇论文戳破了Archetypal SAEs稳定性神话，做可解释性研究的团队需要重新审视自己的评估方法——别被初始化技巧骗了，建议点开看轨迹诊断和消融实验怎么做。
该论文质疑了Archetypal SAEs声称的稳定性优势。研究发现，其稳定性主要源于所有训练运行使用相同的确定性k-means解码器初始化，而非算法本身的约束。作者区分了“稳定性”（独立训练模型间的一致性）和“稳定化”（不同初始化向共同解收敛）两个概念，后者才是可解释性研究真正需要的。当移除共享初始化后，Archetypal约束并未带来额外的稳定化优势。此外，论文还指出预处理依赖的余弦几何问题会干扰终点稳定性指标的解读。研究建议，评估SAE稳定性时应包含轨迹诊断和初始化消融实验。
Ablating Archetypes: The Stability of Archetypal SAEs is an Artifact of Initialization and Metric Design
Dictionary learning with sparse autoencoders (SAEs) produces overcomplete bases from neural network activations that are often interpretable and reduces polysemanticity. However, features from SAEs vary substantially across random seeds -- a problem known as instability. Archetypal SAEs (Fel et al., 2025) were proposed as a general dictionary-learning intervention for more reliable concept extraction, and report more stable dictionaries at the end of training. We demonstrate that the stability claimed by archetypal SAEs is a result of setting identical initialization across multiple runs. Through our analyses, we attempt to clarify two distinct notions in mechanistic interpretability that may be ambiguously used: stability is agreement between two independently trained models, whereas stabilization is the convergence of independently initialized runs toward a common solution. This distinction is critical for mechanistic interpretability of natural language processing (NLP), where feature stability is increasingly used as evidence that SAE features are reusable units of analysis. Experiments from archetypal SAEs share a deterministic k-means decoder initialization, setting inter-run dictionary distance to zero before training begins. When this initialization is removed, the archetypal constraint provides no stabilization advantage in our setting. We further identify a preprocessing-dependent cosine geometry issue that complicates interpretation of endpoint stability metrics. Overall, our study supports the value of studying SAEs within the larger dictionary-learning tradition while showing that stability claims require trajectory diagnostics and initialization ablations.