做神经网络可解释性研究的团队会关心——SAE特征不稳定不是噪声，而是低维结构的基选择问题，看完能帮你更合理设计实验和解读结果。
该研究系统分析了稀疏自编码器（SAE）在不同训练种子下特征的稳定性。研究发现，稳定特征承载了大部分重构和预测相关的信号，而不稳定特征个体不可复现但集中在可复现的低秩子空间中，表明种子依赖性更多反映激活空间内的基模糊性而非纯噪声。通过跨种子聚合独特特征，可以在保持解释方差的同时构建更稳定的SAE。该工作为理解SAE的可靠性提供了新视角。
Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders
Sparse autoencoders (SAEs) are widely used to interpret neural network representations, but their utility depends on whether the learned features are reproducible across training runs. We study this question through \emph{feature stability}: for each SAE feature, we estimate the probability that a similar feature reappears in an independently trained SAE. This yields a scalable per-feature signal that separates stable from unstable features. In a large-scale study across seeds, models, layers, dictionary sizes, and SAE variants, we find a pronounced functional asymmetry: stable features carry most of the reconstruction- and prediction-relevant signal, while unstable features have weak marginal impact and are dominated by low-frequency surface-form triggers in both activation statistics and automatic explanations. Geometrically, unstable features are individually non-reproducible but concentrate in reproducible lower-rank subspaces, suggesting that seed dependence often reflects basis ambiguity within a shared region of activation space rather than pure noise. A controlled synthetic model makes this mechanism explicit, showing that low-rank ground-truth features can be recovered at the subspace level while remaining non-identifiable as individual SAE latents across seeds. Finally, by pooling unique cross-seed features, we construct more stable SAEs while preserving explained variance in this setting. Together, these results show that unstable features are not merely failed or noisy latents: they have weak individual functional impact, but reflect reproducible low-dimensional structure that standard SAEs resolve differently across seeds.