This research provides a new approach to improve the reliability of LLMs in decision-making by calibrating claim-level confidence, which is particularly useful for high-stakes domains where accurate uncertainty estimation is crucial.
This paper introduces claim-level confidence calibration for LLMs, addressing the issue of misaligned confidence with factual correctness. It proposes a framework that decomposes responses into claims and assigns calibrated confidence using inference-time signals. The framework is evaluated on six recent models and shows reduced calibration error on factual questions while revealing failure modes on adversarial false-premise questions.
Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is misaligned with factual correctness. Response-level confidence is a coarse signal: a single generation can mix correct and incorrect statements, so a single number is not actionable for users that must accept, reject, or verify individual pieces of information. We study claim-level confidence calibration as a decision-relevant uncertainty signal: each response is decomposed into atomic, verifiable claims, and each claim is assigned a calibrated confidence using inference-time signals from consistency across samples and self-verification. Our framework operates in closed-box settings (no logits, no fine-tuning) and applies post-hoc calibration directly at the claim level, enabling selective intervention such as evidence retrieval or human review for low-confidence claims. Across TriviaQA and TruthfulQA we evaluate seven baselines on six recent models (Llama-3.1, Mistral, Qwen2.5, DeepSeek-R1, GPT-4, GPT-4o), and show that claim-level decomposition combined with post-hoc calibration reduces expected calibration error on factual questions while exposing failure modes on adversarial false-premise questions where decision-makers most need reliable uncertainty estimates.