论文精选72°

TRACE:跨层证据轨迹纠正减少大模型幻觉,无需训练

TRACE: Trajectory Correction from Cross-layer Evidence for Hallucination Reduction

精选理由

TRACE 解决了大模型幻觉纠正中“一刀切”干预的缺陷,做 LLM 推理优化或事实性研究的开发者可以直接在现有模型上试用,无需额外训练成本。

AI 摘要

TRACE 是一种无需训练的推理时幻觉纠正算法,它通过分析大模型内部各层的候选轨迹,动态选择最合适的纠正方式(如标量反转、早期状态恢复或候选空间修正),而不是固定使用单一干预形式。实验覆盖 15 个模型、8 个模型家族和 3 个事实性基准,所有评估指标均有提升,平均 MC1 提高 12.26 点,MC2 提高 8.65 点,最高提升分别达 47.20 和 43.38 点。该方法无需标签、检索、预训练、微调或逐模型校准。

原文 · arXiv cs.AI

TRACE: Trajectory Correction from Cross-layer Evidence for Hallucination Reduction

Hallucination correction is not a one-direction problem. We show that intermediate layers are neither uniformly more truthful than final layers nor uniformly less trustworthy. Yet hallucination reduction is usually instantiated through one fixed intervention form: contrast one layer against another, steer along a truthfulness direction, or defer to external evidence. This framing is structurally incomplete. Cross-layer factual evidence does not evolve uniformly: in some failures truthful support is present internally and later suppressed, whereas in others candidate competition remains genuinely multi-directional across depth, so no single signed scalar family is generally sufficient. We introduce Trajectory Correction from Cross-layer Evidence for Hallucination Reduction (TRACE), a deterministic, training-free algorithm which corrects hallucinations at inference time by deriving both the corrective layer and the appropriate correction operator from each input's cross-layer candidate trajectory inside the LLM's own forward pass. Under one frozen hyperparameter setting, TRACE selects among scalar reversal, earlier-state recovery, and candidate-space correction using only model-internal evidence. Evaluated as a single universal algorithm across 15 models, 8 model families, and 3 factuality benchmarks, TRACE improves every evaluation cell, yielding mean gains of +12.26 MC1 points and +8.65 MC2-style points with no regressions, with gains reaching +47.20 MC1 and +43.38 MC2-style points. The method uses no labels, retrieval, pretraining, finetuning, or per-model calibration.