这篇论文为LLM控制领域拨乱反正——做模型可解释性、安全对齐或行为调控的团队，值得重新审视稀疏自编码器的潜力，建议点开看看如何用标签修正提升控制效果。
一篇新论文指出，稀疏自编码器作为LLM控制工具表现不佳的结论可能源于特征标签错误，而非方法本身缺陷。早期研究因标签与模型内部实际因果行为不匹配，导致稀疏自编码器看起来效果差。作者提出监督式管道，通过验证特征活动是否可靠追踪真实数据标签来替换模糊标签，并发现高稀疏性并非必要。尽管提示工程仍更强，但特征控制可直接操纵模型内部机制，为模型行为调控提供新思路。
The paper argues that sparse autoencoders may not …
The paper argues that sparse autoencoders may not be bad steering tools after all, and much of the earlier failure may have come from choosing and naming the wrong features.
The problem is that earlier work made sparse autoencoders look weak because their features were labelled in a way that may not match what those features actually cause inside the model.
A sparse autoencoder is a small helper model that breaks an LLM’s hidden activity into many possible “features,” such as a topic, style, or concept.
So a sparse autoencoder finds directions inside a model, but an unnamed direction is not yet a usable control knob.
The authors replace vague or inherited labels with a supervised pipeline that asks whether one feature’s activity reliably tracks a real label in data.
As to the mechanism, if a feature fires on “alcohol,” and forcing that feature upward makes the model talk about alcohol, the label is no longer just descriptive; it has causal weight.
The paper also finds that very high sparsity may not be necessary, meaning the feature does not need to be extremely rare to be useful for steering.
Also to note here, both prompting and feature steering are ways to push an LLM toward a desired behavior.
Prompting remains stronger because the model was trained to obey prompts, while feature steering is more like pressing directly on the machinery and hoping the rest stays intact. Prompting says “write about alcohol” in the input; feature steering instead turns up the model’s internal “alcohol-related” feature and sees whether the output changes in that direction.
----
Link – arxiv. org/abs/2605.31183
Title: "Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines"