这篇论文用新指标发现，量化后的模型即使分数不变，行为也可能跑偏，搞模型部署的同学建议看看。
该论文提出了正确性一致性（correctness agreement）指标，用于衡量基础模型与量化变体在正确预测上的重叠程度。在8-bit至2-bit的多种量化方案下，发现即使任务性能看似保持，中等量化也会导致行为分歧。分析表明查询和键投影的敏感性高于值和输出投影，揭示了低比特宽度的非线性断点。这些发现指出仅依赖准确率和困惑度会掩盖量化的真实影响。
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
Post-training quantization is widely used to deploy large language models in resource-constrained settings, yet its evaluation relies almost exclusively on accuracy and perplexity. We show that these metrics fail to capture behavioral changes induced by quantization. We introduce correctness agreement, a decision-level metric that measures overlap in correct predictions between a base model and its quantized variants, independent of absolute accuracy. Across multiple models and quantization schemes from 8-bit to 2-bit, we find that behavioral divergence emerges under moderate quantization even when task performance appears preserved. To explain this effect, we analyze quantization as a structural operator on attention weights and quantify layer-wise distortions using statistical and distributional measures. Our results reveal non-linear breakpoints at low bit-widths and show that query and key projections are consistently more sensitive than value and output projections. These findings expose an illusion of equivalence between base and quantized models and motivate behavioral evaluation beyond conventional performance metrics.