Qwen3 小模型上量化精度与答案顺序平均的算力分配研究
Bits or permutations? Spending a fixed inference budget on weight precision or on option-order averaging in small local language models
两个 Qwen3 小模型上的实测账本：6 bit 量化不掉分，4 bit 会放大选项位置偏差。结论很实用——用 6 bit 权重，省下的预算拿去做最多四次选项顺序平均。做本地部署和评测的人可以直接抄这张决策表。
一篇 arXiv 论文研究了本地小模型在固定推理预算下，应把开销花在权重精度还是答案选项顺序平均上。实验基于 Qwen3-0.6B 和 Qwen3-1.7B，模拟 round-to-nearest 量化（8、6、4、3 bit，fp16 参照），在 ARC-Easy、ARC-Challenge、MMLU、HellaSwag 各 300 题上读取 A-D 选项 logits。结果显示 6 bit 相对 fp16 不掉分（0.6B 反而 +1.8 点，1.7B +0.9 点），8 bit 变化 -0.7 和 -0.4 点；4 bit 时答案位偏斜从 0.15 升到 0.53（0.6B）、0.07 升到 0.12（1.7B）。
机制上，论文用等功率高斯噪声做对照，发现偏斜上升是否超出同等噪声水平尚无定论；翻转率虽上升但低于精度匹配的独立均匀错误基线。循环平均四个选项顺序在 4 bit 下可挽回 12.6 点（0.6B），fp16 下为 +5.3 点。
作者用"权重字节读取数乘以遍数"这一机器无关单位给 bits 和 passes 定价，成本配对比较后的结论是：仅当权重量化到 6 bit 及以上时，加顺序优于加精度；决策表简化为使用 6 bit 权重并把剩余预算花在最多四个顺序、每顺序一遍。局限是仅在两个小模型和最简单量化器上模拟，作者检索未发现此前将量化与选项顺序联合研究的工作。
Bits or permutations? Spending a fixed inference budget on weight precision or on option-order averaging in small local language models
Local language models can save memory by storing weights in fewer bits, and they can reduce sensitivity to the order of answer options by averaging a few reorderings. We ask whether the first makes the second worse, and which is the cheaper way to spend a fixed budget. On Qwen3-0.6B and Qwen3-1.7B with simulated round-to-nearest weight quantization (8, 6, 4 and 3 bits, fp16 reference), read from the logits of the letters A to D on ARC-Easy, ARC-Challenge, MMLU and HellaSwag (300 items each), 6 bits lose no accuracy against fp16 (they score +1.8 (+1.0 to +2.5) and +0.9 (+0.3 to +1.5) points higher, which we do not explain), and 8 bits change it by -0.7 (-1.1 to -0.3) and -0.4 (-0.7 to -0.2) points. At 4 bits the answer-slot skew rises from 0.15 to 0.53 (0.6B) and from 0.07 to 0.12 (1.7B); Gaussian noise at the same weight SNR gives 0.15 and 0.37 (two draws) and 0.19 (one draw), so the control is inconclusive on whether rounding adds slot skew beyond equal-power noise. Flip rates rise but stay below an accuracy-matched baseline of independent uniform errors. Averaging four cyclic orders recovers +12.6 points at 4 bits for 0.6B (+5.3 at fp16). We price bits and passes in one machine-independent unit, weight bytes read times passes. In cost-matched pairs, extra orders beat extra bits only when the weights have 6 or more bits, and the decision table reduces to: use 6-bit weights and spend the rest of the budget on up to four orders, one pass each. Everything is simulated on two small models with the simplest quantizer; a title-level search found no earlier study of quantization together with option order.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Qwen3, 量化, 本地模型, 推理优化.
Source attribution: curated from arXiv cs.AI (@Mohammed Nayeem Khan, Logesh L, Aliyas Ahmad Misri).
