这篇论文教你用统计方法严格证明LLM有多安全，不是估算，而是可证明的下界。
本研究提出一种新框架，使用Clopper-Pearson置信区间为LLM生成的有害输出概率计算PAC下界。该算法通过分析潜在空间特征优先探索自回归生成树中更可能产生有害输出的分支。该方法能高效计算极小有害概率（如低于10^-6）下的严格下界。实验在多个SOTA LLM上验证了有效性，所得下界经形式证明小于真实有害概率。
Sound Probabilistic Safety Bounds for Large Language Models
We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt. We study a new application of the Clopper-Pearson confidence intervals to obtain probably approximately correct (PAC) bounds for this problem. As our main technical contribution, we propose an algorithm that leverages features in the latent space to prioritize exploring branches in the auto-regressive generation tree that are more likely to produce harmful outputs. Our approach in particular enables the efficient computation of useful lower bounds, even in scenarios where the true harm probability is extremely small, and crucially, the obtained lower bounds are sound, i.e., formally proven to be less than the actual harmfulness probability: our experimental results demonstrate the effectiveness of our method by computing non-trivial lower bounds on state-of-the-art LLMs. This study newly enables the evaluation and statistical certification of LLMs.