想治治AI裁判的偏袒病？这论文用数学几何直接纠偏，不用重训模型，比调prompt靠谱多了。
该研究将LLM作为评判者时的语速偏见等系统性偏差定义为问题核心。作者将有限人类监督下的LLM评估建模为正-无标记学习问题。提出基于部分最优传输(Partial Optimal Transport)的几何审计框架，无需重新训练即可识别人类一致偏好并纠正有偏评判者。实验表明该方法在提升与人类偏好一致性、增强对呈现偏差鲁棒性上优于现有流水线，并提供可解释的置信度估计。
Quantifying and Auditing LLM Evaluation via Positive--Unlabeled Learning
Large Language Models (LLMs) are increasingly used as judges for scalable evaluation, yet such LLM--as--a--Judge systems exhibit systematic biases that are decoupled from semantic quality, most notably verbosity bias. Meanwhile, human supervision is costly and typically selective, yielding reliable positive judgments but leaving most outputs unlabelled and potentially mixed in quality. We formulate LLM evaluation under selective human supervision as a positive--unlabelled learning problem and propose a geometric auditing framework based on Partial Optimal Transport. By aligning a small set of human--verified positives with a reliable subset of unlabelled outputs in a fixed embedding space, our method identifies human--consistent preferences and corrects biased judges without retraining. Experiments demonstrate improved alignment with human preferences, increased robustness to presentation biases, and interpretable confidence estimates, offering a scalable and statistically grounded alternative to existing LLM--as--a--judge pipelines.