这篇论文发现了LLM回答重复问题时信念会自己稳定，还给了两种让模型更一致的方法，适合关注推理可靠性的读者。
论文发现LLM在多项选择问答中早期存在信念漂移，违背鞅性质。通过提出的提示预测重采样（PPR）方法，模型在多次重采样后信念自稳定并收敛。基于此，研究者进一步提出种子答案提示策略和自一致性损失微调方法。在多项选择QA基准测试中，这些方法显著减少信念漂移并提高预测一致性，且不牺牲准确性。
From Drift to Coherence: Stabilizing Beliefs in LLMs
Large language models (LLMs) are often hypothesized to perform implicit Bayesian inference, yet a key coherence condition, the martingale property of predictive beliefs, has been shown to fail in controlled synthetic in-context learning settings. We revisit this question in a more typical usage regime: generic multiple-choice question answering. Exploiting the discrete answer space, we compute exact predictive distributions and study belief dynamics induced by autoregressive answer resampling. We introduce prompted predictive resampling (PPR), where an LLM generates a sequence of answers to the same question. Empirically, PPR reveals early-stage belief drift, indicating martingale violations. However, after sufficient resampling steps, the belief process self-stabilizes and converges to a coherent predictive distribution. Based on this observation, we further propose (i) a seed-answer prompting strategy to accelerate stabilization, and (ii) a self-consistency loss that amortizes early-stage drift into the model via fine-tuning. Experiments on multiple-choice QA benchmarks show that our methods substantially reduce belief drift and improve predictive coherence without sacrificing accuracy.