arXiv 论文：小模型按步动态升精度解码，Qwen3-0.6B 在 GSM8K 上验证
Precision on Demand: Escalating Weight Bits at Uncertain Decoding Steps in a Small Local Model
这篇论文尝试让小模型平时用 4-bit 算，只在不确定的步骤回退 8-bit，代价测算得很细。诚实地说，实机数据还没和全精度拉开显著差距，但门控捕获率和字节成本分析对做端侧推理量化的人很有参考价值，适合 Qwen3 用户和量化工程方向的研究者看。
一篇 arXiv 论文（2611.14418）提出在本地解码时不再全程固定权重精度：权重以 8-bit 存储，平时只用前 4-bit 计算，当低精度输出分布不确定（熵超阈值或 top-two 差距很小）时才回退到 8-bit 重算该步。实验在 Qwen3-0.6B 上以 GSM8K 为基准，成本按每 token 读取的权重字节数计算。
在 36724 个回放步骤上，低精度 pass 的熵预测 argmax 翻转的 AUC 达 0.93；15% 升级预算下，熵门控捕获 78% 的翻转，margin 门控捕获 94%，而同预算随机门控的期望仅为 15%。150 道实机题中，全高精度得分 64.0%，全低精度 55.3%（McNemar p=0.053）；16% 升级率的熵门控得 62.7%，只消耗全高精度 0.87 的字节数。
作者坦承局限：其实测配对差异为 -1.3（-8.0 到 +5.3）点对全高精度、+7.3（+0.0 到 +14.7）点对全低精度（p=0.080），实机数据未能与两端统计显著区分；同成本模型下经验证的推测解码可达到零 KL。未做实机随机对照、thinking 模式相位规则、第二模型和实机 KL 实验。
Precision on Demand: Escalating Weight Bits at Uncertain Decoding Steps in a Small Local Model
Local decoding usually fixes one weight precision for the whole generation. We study a different allocation: store 8-bit weights, decode with their top 4 bits, and redo a step at 8 bits only when the low-precision output distribution looks uncertain (entropy above a threshold or a small top-two margin). Quantization is simulated, the model is Qwen3-0.6B, and the benchmark is GSM8K. Cost is counted in weight bytes read per token. On 36724 replayed steps the entropy of the low pass ranks argmax flips with AUC 0.93. At a 15% escalation budget it catches 78% of flips and the margin gate 94%, against the 15% expected for a random gate of equal rate (a computed expectation, not a run). In 150 live problems all-high scores 64.0% and all-low 55.3% (McNemar p=0.053). An entropy gate with a 16% escalation rate scores 62.7% at 0.87 of the all-high weight bytes. Its paired difference is -1.3 (-8.0 to +5.3) points to all-high and +7.3 (+0.0 to +14.7) points to all-low (p=0.080), so the live data do not separate the gate from either end. Verified speculation reaches zero KL at similar bytes in the same cost model. A live random control, a thinking-mode phase rule, a second model and live KL were not run.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Qwen3, 量化, GSM8K, 推理优化.
Source attribution: curated from arXiv cs.AI (@Mohammed Nayeem Khan, Logesh L, Aliyas Ahmad Misri). Cross-source corroboration: 3 independent sources have covered this development (including 宝玉, IT之家), confirming industry-wide signal strength.
