#numerical precision

4 stories · 1 in 7 days · 2 in 30 days
Stories, products and papers tagged "numerical precision", newest first.
Nov 17
Chinese content微调能换来多少解码确定性?边际正则化抑制批处理导致的 token 翻转

一篇挺有意思的研究:贪心解码其实不是确定性的,同一 prompt 换个 batch 就可能输出不同 token。作者用 logit 边际正则化在 GSM8K 上把发散率从 53.8% 压到 38.1%,代价 1-3 个准确率点。做需要可复现输出的服务或评测的同学可以看看。

arXiv cs.AI@Charan Teja Nagisettygari, Siva Raja SindiramuttySource
Oct 30
Sep 23
PapersBuildersSelected71°10:01
Precision Choices Break Greedy Decoding Determinism in LLMs

Anyone chasing reproducible LLM outputs — evaluation harness authors, serving teams and quantisation engineers — should treat BF16 vs FP16 as a real source of run-to-run variance, not an implementation detail. The paper offers a low-overhead fix (threshold-triggered FP32 LM head recomputation) and maps when it fails, including FP8 end-to-end serving. The margin-based analysis also gives practitioners a quick diagnostic for whether their own deployments are at risk.

arXiv cs.LG@Gaoyuan Du et al. (7)Source
Jul 31