这篇论文给出了一个白盒方法ROBIN，能精准定位模型中的偏见注意力头并修复，比直接抹掉整个头效果好。
研究者提出ROBIN方法，通过敏感度分析排序注意力头并移除偏见子空间。在四个模型的实验中，ROBIN在所有模型上降低了WinoBias差距，同时保持了语言建模质量，优于整体置零方法。这些初步结果表明头级偏见修复不仅要考虑选择哪些头，还要考虑如何修改。
Toward Localizing and Repairing Bias in Transformer Attention Heads
Transformer language models are increasingly used as software components, yet biased outputs remain difficult to localize and repair inside the model. Existing fairness testing and repair methods largely operate at the input-output or retraining level, while recent work suggests that bias-related behavior can concentrate in a small set of attention heads. This paper studies whether attention heads can be localized and repaired through a targeted inference-time intervention. We introduce ROBIN, a white-box head-level fairness debugging method that ranks attention heads using sensitivity to fairness probes and removes a small bias subspace from selected head outputs. In a four-model pilot study, ROBIN reduces the measured WinoBias gap across all models while preserving language-modeling quality better than whole-head zeroing. These preliminary results suggest that head-level bias repair should consider not only which heads are selected, but also how selected heads are modified.