Latest

Every published item, in order · 8120 items
Oct 20
PapersBuildersSelected09:50
Statistical Framework Unifies Adam, Shampoo and Muon Optimizers

The practical value lies in the unification: one statistical principle explains Adam, Shampoo and Muon, so practitioners can tune or combine these optimizers with a clearer theoretical map instead of treating them as folklore. Engineers training GPT-scale models get concrete alternatives, since MND beats Distributed Shampoo and KL-Shampoo on stability and runtime, and MND-Muon wins 21 of 24 paired runs against Muon. Researchers in optimizer design can extend the covariance-approximation framework to new preconditioners.

arXiv cs.LG@Tetiana Parshakova et al. (3)Source
PapersBuildersMulti-source09:50
Chinese content研究显示小模型的多语言效率常被英文基准掩盖:印度语系分词器覆盖不足

这篇论文用数据戳破一个常见假设:小模型“多语言省内存”的说法对印度语系根本不成立。卡纳达语用户只有英文用户 6% 的上下文,数字很扎心,做本地化的开发者应该看看。

Event
arXiv cs.LG@Anna PramodSource
Tomorrow 08:00, the daily lands in your inboxTelegram channel or email. Same screening as this page, no ads.
Chinese contentPAA-Bench 提出重新评估 VLM 视觉 token 剪枝:准确率和 token 数都不是好指标

论文发现现行的 VLM 剪枝评测方式有漏洞,模型可能靠文本捷径蒙混过关。作者做了个 PAA-Bench 框架和 PCS 指标,8 种方法实测后发现压缩更多 token 未必更快,想给剪枝方法做评测的可以看看。

arXiv cs.AI@Geng Li et al. (4)Source
Oct 19
PapersBuildersMulti-sourceSelected12:18
FuTransformer: FP8 prefill and INT4 decode kernels for Hopper GPUs

The paper shows the quality-price of aggressive quantization is now measured, not guessed: INT4 weights plus a Hadamard-rotated INT4 cache cost under one perplexity point while roughly tripling decode speed on H100. Engineers serving Llama-3.1-class models on Hopper GPUs can use its fragment-layout dequantization and KV-cache placement techniques to cut latency without falling back to padded prefill. Compared with PyTorch's INT4 kernel, which loses to BF16 at batch 32, this design stays fast up to batch 32.

Event
arXiv cs.AI@Wei WangSource

Only the latest 2000 items are listed; for older ones see the daily archive