TopKLoRA：可解释稀疏微调方法可定位并消除模型后门
Interpretable sparse fine-tuning with TopKLoRA enables faithful circuit discovery
用 TopKLoRA 微调出的回路比普通 LoRA 小十倍，还能精准切除后门而不伤指令能力，做模型安全和可解释性研究的值得看代码。
论文提出可解释性设计优先的 LoRA 变体 TopKLoRA，每个 token 只激活 k 个最大的 adapter latents，仅用下游损失训练，不依赖 SAE。团队用它在 gemma-2-2b、Qwen2.5-1.5B 和 Qwen2.5-7B 上制作 sleeper-agent 后门模型，对每个模型都找到并消融了实现后门的必要且充分回路。相比稠密 LoRA 基线，TopKLoRA 产生的回路规模最多小一个数量级，能在不损失指令跟随能力的情况下外科手术式移除恶意行为。代码已在 GitHub 开源。
Interpretable sparse fine-tuning with TopKLoRA enables faithful circuit discovery
Fine-tuning on datasets of unknown origin can create a backdoor in the trained model when the data includes poisoned examples. Difficult to localise and remove, backdoors can persist through safety training. While parameter-efficient fine-tuning methods capture the full parameter changes introduced during training, they are not easily interpretable. We present the full method of TopKLoRA, extending the preliminary proposal of Masiak et al. (2025), an interpretable-by-design LoRA variant in which only the $k$ largest adapter latents are active at each token. Unlike SAEs, TopKLoRA is trained purely on downstream loss, where optimisation pressure incentivises learning functionally causal latents. We demonstrate this sparse LoRA adapter in a sleeper-agent backdoor case study where we create model organisms by fine-tuning gemma-2-2b, Qwen2.5-1.5B, and Qwen2.5-7B on a poisoned instruction-following dataset. For each organism, we identify and ablate a circuit that is both necessary and sufficient to implement the backdoor. Furthermore, our new fine-tuning method produces sparse circuits that are up to an order of magnitude smaller than a dense LoRA baseline, allowing surgical ablation of the malicious capability without sacrificing instruction-following performance. Finally, we propose a technique to construct model organisms with localised circuits using TopKLoRA latents and gradient-routing, which we use to validate the findings of our circuit search algorithm. We consider these findings as a step toward interpretable-by-design architectures that provide faithful units of analysis without compromising model performance. We share our code: https://github.com/interpretable-finetuning/TopKLoRA.