Papers

论文审计:注意力头偏见定位方法统计上不可辨识

What Does a Bias Head Measure? Non-Identifiability in Head-Level Bias Attribution

Why we picked it

研究直接拆了“找到某个注意力头、删掉就能去偏见”这类论文的方法根基:头映射其实在测提示词差异而非偏见。做模型公平性、可解释性或安全审计的人应该读,能避开一整类不可信的去偏见实验设计。

一项针对偏见定位方法的审计研究在 Qwen-2.5-7B-Instruct 和 Llama-3.1-8B-Instruct 上展开,发现按注意力头定位社会偏见的主流做法存在根本问题。头级统计量与仅看回复长度的映射在两个模型上相关系数分别达到 ρ=-0.950 和 -0.768,表面看像是长度混淆因素。

Body

但干预实验否定了这一解读:固定提示词后,回复长度只解释统计量方差的 1%,而提示词本身解释 64%,且排序与长度并非单调关系。在同一提示词内,公平性标签、回复长度和保护群体标记词共线性达 φ=0.95,部分源于标注规则的定义,因此任何观察性分析都无法把头映射归因于其中任何一个因素。

实质性结论同样不成立:在两种参考解读下,偏见选择性在两个模型上都不是集中于后期层,作者测试的所有头集合消融后均未降低不公平率。互补方向的干预则可靠地破坏拒绝行为(p=2.7×10⁻⁵),说明该流程本身有能力检测这个量级的效应,问题出在定位假设而非工具灵敏度。

Original · arXiv cs.LG

What Does a Bias Head Measure? Non-Identifiability in Head-Level Bias Attribution

Methods that localise social bias in large language models to individual attention heads and remove it by ablation are validated by their effect on a fairness metric, never by evidence that the statistic selecting the heads measures bias. We audit this on Qwen-2.5-7B-Instruct and Llama-3.1-8B-Instruct. The head map correlates at $ρ= -0.950$ and $-0.768$ with a label-blind response-length map, apparently identifying a surface confound. An intervention that breaks the length-label coupling rejects this reading: with the prompt held fixed, response length explains 1% of the statistic's variance. What does explain it is prompt identity, at 64% of variance, in an ordering that is not monotonic in length. Within any fixed prompt the fairness label, response length and the presence of a protected-group token are collinear at $φ= 0.95$, partly by definition of the labelling rule, so no observational analysis can attribute the map to any of them, and no method in this family performs an experiment that could. The substantive claims fail independently: bias selectivity is not late-layer-concentrated on either model under either reference reading, and no head set we tested reduces the unfairness rate when ablated. Its complement, scored on the opposite label, reliably destroys refusal behaviour ($p = 2.7\times10^{-5}$), demonstrating that the pipeline detects effects at this scale.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Qwen, Llama, interpretability, AI safety.

Source attribution: curated from arXiv cs.LG (@S. M Seefat Alam).