BiG-SURE能帮你评估黑盒大模型的不确定性，无需访问模型参数，简单实用。
研究人员提出BiG-SURE，一种基于跨温度语义一致性的不确定性估计方法。该方法通过构建锚点-探针二分图，使用NLI蕴含分数计算置信度。在文本QA、多语言QA和多模态QA任务中，BiG-SURE在黑盒模型设置下提升了平均弃权AUROC。
BiG-SURE - Bipartite Graph for Semantic Uncertainty and Reliability Estimation of LLMs
Reliable uncertainty estimation is a crucial requirement for deploying large language models (LLMs) and vision-language models (VLMs) in safety-critical settings, especially when the model parameters are not accessible (black-box). We propose BiG-SURE, an uncertainty estimator based on cross-temperature semantic agreement. The method samples low-temperature responses as stable semantic anchors and high-temperature responses as probes under meaning-preserving input transformations. It then constructs an anchor-probe Bipartite Graph (BiG) using NLI-based entailment scores and defines confidence through the normalized squared spectral energy of this matrix, with uncertainty given by its complement. This bipartite graph-based Semantic Uncertainty and Reliability Estimation (SURE) score measures whether high-temperature probes remain semantically aligned with the model's stable low-temperature belief or not. We evaluate BiG-SURE on text QA, multilingual QA, and multimodal QA tasks across multiple model families. In these experiments, BiG-SURE improves average abstention AUROC over prior black-box uncertainty estimators, while remaining simple, unsupervised, and applicable to black-box model settings.