这篇论文用几何视角解释LLM评判偏见，发现偏见隐藏状态的低维子空间，还能预测模型在未知基准上的失败，挺有意思。
该论文从机制可解释性角度研究LLM作为评判者的评分偏见，覆盖7种评判模型、7种偏见类型和9个基准。研究发现偏见输入在隐藏状态中沿低维子空间位移，且该子空间在深层网络中更为显著。通过沿该子空间进行因果控制，可在清洁输入上复现偏见过打分，在偏见过输入上恢复基线打分。线性投影到偏见方向特征可预测模型在3个未见基准上的失败，效果远超基于文本的替代方法。
Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge's hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts reproducing biased scoring on clean inputs and reverse shifts restoring baseline scoring on biased ones, while matched-norm random directions produce shifts an order of magnitude smaller. Operational: a simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks, substantially outperforming text-based alternatives. Reading bias as activation geometry, rather than as input-output noise, unifies geometric structure, causal control, and operational prediction within a single framework. The project page is available at https://xzx34.github.io/unfair-judge/