论文用实验告诉你，4比特量化的LLM最靠谱，比8比特还稳，想省资源又想要可靠性可以照这个选。
该论文系统评估了量化LLM在2、3、4、8比特位宽下的可靠性，涵盖不确定性、校准和鲁棒性三个维度，使用了六种不同量化方法。研究发现，模型性能随总比特数单调提升，但可靠性缩放呈非线性，4比特量化模型达到可靠性峰值。实验还表明，量化增强了LLM对字符级和词级自然扰动的鲁棒性。
Reliability Scaling Laws for Quantized Large Language Models
Quantization is a powerful strategy to build capable and resource-efficient large language models (LLMs) by reducing the bitwidth of the parameters. While quantized LLMs achieve state-of-the-art performance on unperturbed inputs using standard predictive metrics, their performance on perturbed inputs, measured using reliability metrics, remains underexplored, despite its importance for reliable deployment. To address this gap, we first conduct a comprehensive reliability evaluation of quantized LLMs consisting of three key components: (1) Uncertainty: We assess the trustworthiness of LLMs quantized to 2, 3, 4, and 8 bits using six different quantization methods, employing established uncertainty metrics. (2) Calibration: We assess how well-calibrated the uncertainty estimates of quantized models are across model scales and bit precisions. (3) Robustness: We design character-level and word-level input perturbations to evaluate the reliability of quantized models under semantically-preserving variations in the inputs that arise in real-world applications. Second, we characterize how reliability scales with the total number of model bits. Our study reveals that while the performance scales monotonically with the total number of bits, the reliability scalings are nonlinear. A reliability peak occurs for 4-bit quantized models, indicating that quantizing moderately sized models offers the best reliability-efficiency trade-off. Additionally, our empirical findings reveal that quantization enhances the robustness of LLMs to natural input perturbations.