研究显示 BLEURT 构造幻觉标签不可靠，误标率高达 47%
When Ground Truth Is Not Ground Truth: Automatic Labeling for Hallucination Detection
做幻觉检测评估的可以看看：用 BLEURT 相似度打标签误标率近一半，QGen-9B 只有 12.86%，选 judge 前先读这篇。
一篇 arXiv 论文研究了幻觉检测中用 BLEURT 等相似度指标自动生成 ground-truth 标签的做法。实验覆盖 15 个自动监督 judge、7 个检测基线、2 个生成模型、2 个 QA 数据集和 5 种语言。人工验证显示 BLEURT 和 BLEURT-n 的误分类率达 47.23% 和 47.32%，而 QGen-9B 仅 12.86%。论文指出基于阈值自动构造标签的方式在事实性判断上存在明显缺陷。
When Ground Truth Is Not Ground Truth: Automatic Labeling for Hallucination Detection
Hallucinations in Large Language Models (LLMs) remain a major challenge, as they can produce misleading, factually incorrect, or unverifiable information, thereby undermining their trustworthiness and reliability. However, reliable ground-truth labels remain a key challenge in evaluating hallucination detectors. Recent studies often construct labels automatically by comparing model generations with reference answers using similarity metrics such as BLEURT. This implicitly treats BLEURT similarity as a proxy for hallucination, despite its English-centric supervision, unbounded score range, and design as a similarity metric rather than a factuality measure, making threshold-based ground-truth construction potentially unreliable. Through comprehensive experiments using 15 automatic supervision judges, seven hallucination detection baselines, two generator models, two QA datasets, and five languages, we demonstrate that detector performance varies substantially with the choice of supervision judge. Human validation reveals substantial differences in labeling reliability, with QGen-9B achieving the lowest overall misclassification rate (12.86%), while BLEURT and BLEURT-n exhibit rates of 47.23% and 47.32%, respectively. Our findings highlight the limitations of automatic similarity-based labeling and the need for more reliable judging methods that better capture semantic and factual correctness.