这篇论文用FEGA方法分析了稀疏自编码器特征的下游几何，发现大多数特征不是单方向，value-like和pointer-like特征效果不同，对理解模型可解释性很有启发。
稀疏自编码器（SAE）作为可解释性工具，常因特征与模型行为间链接不一致而受限。论文提出特征效应几何分析（FEGA）框架，通过跨上下文移除相同活跃SAE特征并分析logit变化云。发现一致的一维效应罕见，仅有少数特征像可重用方向。将特征分为value-like（静态信息）和pointer-like（上下文相关操作），前者更多表现出结构化低维效应，后者则呈现扩散效应。研究表明，特征虽可解释且有因果关联，但未必提供稳定操控方向。
Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects
The wide-scale use of sparse autoencoders (SAEs) as interpretability tools is limited by inconsistent links between SAE features and model behavior. Features with clear activation descriptions may have weak or unexpected causal effects; steering can vary across prompts or oppose the intended direction; and activation-based feature selection can miss features that produce the desired output change. Prior work has studied feature geometry inside the model, where features are computed. We instead study the geometry of changes in model logits caused by feature interventions. We introduce Feature-Effect Geometry Analysis (FEGA), an unsupervised framework that removes the same active SAE feature across contexts and analyzes the resulting cloud of logit changes. Across SAE variants, consistent one-dimensional effects are rare: few features behave like reusable directions. To interpret this variation, we distinguish value-like features, tied to static information such as factual attributes, from pointer-like features, associated with context-dependent operations. Value-like features more often exhibit structured, low-dimensional effects, although these effects typically span several directions. Pointer-like features, by contrast, predominantly exhibit diffuse effects. Our results show that a feature can be interpretable and causally relevant without providing a stable direction for steering.