Latest

Every published item, in order · 9845 items
Nov 9
PapersBuildersMulti-sourceSelected13:09
BitDecoding-FP4 Cuts KV Cache to FP4 With 2.4-3.0x Throughput Gains

The co-design removes the trade-off that previously forced a choice between quantization accuracy and decode speed: system-level changes like transposed softmax eliminate lookup overheads that plagued earlier FP4 KV methods. Teams running long-context inference on NVIDIA Blackwell hardware can apply it directly, since the implementation is open-sourced and works across dense and MoE models with GQA and MLA, and it even holds up in RL training rollouts.

Event
arXiv cs.AI@Dayou Du et al. (6)Source
Tomorrow 08:00, the daily lands in your inboxTelegram channel or email. Same screening as this page, no ads.
PapersBuildersMulti-sourceSelected12:56
RidgeFill Retrofit Speeds Prefill on Qwen and Llama Models

The non-obvious point is that YOCO-style self-forecasting caches, previously requiring dedicated training like DeepSeek-V4.1-Flash, can now be retrofitted onto off-the-shelf checkpoints with no training and no dropped tokens. Engineers deploying long-context inference in llama.cpp or vLLM can use it to cut time to first token on a single A100. Benchmarks like RULER and LongBench v2 let readers judge the quality cost themselves.

Event
arXiv: DeepSeek@Dylan Johnston, Robert P. SmithSource

Only the latest 2000 items are listed; for older ones see the daily archive