这篇论文为SAE的可解释性提供了理论根基，做可解释AI或模型控制的开发者可以直接参考其结论来设计更可靠的SAE变体。
稀疏自编码器（SAE）在解析神经网络表征为可解释概念方面取得了成功，但其提取内容的科学结论尚不明确。本文避开传统的数据生成模型，直接研究字典学习最优解必须满足的性质。作者将局部最优性分析扩展到非负联合优化问题，推导出最优SAE特征与其分布之间的约束关系。这些约束解释了SAE的多种行为，包括层次分裂与吸收、残差结构以及密集对跖特征。最后，作者构建了一个新的大字典凸问题，探索了每个数据点对应大量原子的极限情况，为设计下一代SAE提供了理论指导。
How Optimality Structures Sparse Dictionaries: A Theory for Understanding SAE Representations
Sparse Autoencoders (SAEs) have found success parsing neural representations into interpretable concepts, providing a basis for understanding and control. However, what exactly SAEs extract, and, correspondingly, the scientific conclusions we can draw from them, are not obvious. Empirically, the proof is in the pudding: SAEs learn interpretable features. Theoretically, we lack a clear account of what properties a 'concept' must satisfy for an SAE to extract it. There has been extensive identifiability work studying the conditions under which sparse coding recovers ground-truth features; however, these approaches tends to focus on simple data-generating models (e.g. sparse independent features) which poorly approximate the internet-swallowing language-model representations on which SAEs are trained. Here, avoiding data-generating models, we ask simply what properties any dictionary learning optimum must satisfy. Concretely, we extend local optimality analyses (Gribonval & Schnass, 2010) to the nonnegative joint-optimisation problem that vanilla SAEs approximate, and derive constraints relating optimal SAE features to their distributions. We use these constraints to explain a range of observed SAE behaviours - hierarchical splitting & absorption, the structure of residuals, and dense antipodal features - each reflecting how L1+nonnegativity interact with data to structure optimal dictionaries. Finally, we construct a novel large-dictionary convex problem and explore the wide atom-per-datapoint limit. In sum, we hope to tease model assumptions from unexpected observations, letting us learn more from SAEs' successes and provide principles for designing their successors.