这篇论文实验发现，对评估器做概率校准能把偏好耦合降低20-49%，效果显著，推荐给做LLM评估管线的朋友。
该论文首次研究评估器校准作为缓解方法，对DeepSeek-V4-Pro执行器和GLM5.2评估器进行受控实验。结果显示，校准将耦合系数gamma降低20-49%，Jensen-Shannon散度降低45-67%。对称LR控制实验确认效果并非来自更新不对称性减少。作者发布了校准TTRL协议，推荐作为轻量级缓解方案。
Calibrating the Evaluator: Does Probability Calibration Mitigate Preference Coupling in LLM Agent Feedback Loops?
When large language model (LLM) agents adapt their behavior through evaluator feedback, systematic evaluator biases propagate into the agent's learned strategy distribution - a phenomenon termed evaluator preference coupling. Prior work has documented this coupling and established a diagnostic framework (EPC) to measure it, but has not investigated whether calibration techniques can mitigate the effect. We present the first study of evaluator calibration as mitigation: applying probability calibration to the evaluator's pairwise judgments to reduce spurious preference propagation. In a controlled within-subjects experiment (N=5) comparing standard binary TTRL (win/loss) with confidence-calibrated TTRL (probability-weighted updates) using DeepSeek-V4-Pro as executor and GLM5.2 as evaluator, we find that calibration reduces the coupling coefficient gamma by 20-49% and Jensen-Shannon divergence by 45-67%. A symmetric-LR control confirms the effect is not due to reduced update asymmetry. We release the calibrated TTRL protocol and recommend it as a lightweight mitigation for LLM-as-judge deployment pipelines.