这篇论文提出了CircuitLasso，能以更低成本达到和现有方法一样好的电路学习效果，还能揭示可解释的语义特征如何传播。
论文针对LLM电路学习中原始神经元多义性和SAE特征高维度的计算瓶颈，提出基于稀疏线性回归的CircuitLasso方法。在基准数据上，CircuitLasso恢复电路的结构准确性与最先进的干预方法相当，但计算成本大幅降低。它还能高效揭示SAE特征之间的传播关系，展示可解释语义特征如何影响模型预测。在领域泛化任务中，利用CircuitLasso学到的电路洞见，能以更低成本达到可比性能。
Scalable Circuit Learning for Interpreting Large Language Models
A prominent research direction in mechanistic interpretability is learning sparse circuits over LLM components to reveal how they jointly produce model behavior. However, raw neurons are polysemantic, making learned circuits hard to interpret. Sparse autoencoder (SAE) features alleviate this, but their high dimensionality makes existing intervention-based circuit learning methods computationally prohibitive. We propose CircuitLasso, a scalable circuit-learning approach based on sparse linear regression. CircuitLasso recovers circuits whose structural accuracy matches that of state-of-the-art intervention-based methods on the benchmark data, at a fraction of the computational cost. For interpretability, CircuitLasso efficiently uncovers relationships among SAE features, showing how human-interpretable semantic features propagate through the model and influence its predictions. Finally, we validate the utility of our learned circuits by leveraging their insights to achieve comparable performance at substantially lower cost on a domain-generalization task.