Latest

Every published item, in order · 9845 items
Nov 4
Tomorrow 08:00, the daily lands in your inboxTelegram channel or email. Same screening as this page, no ads.
PapersBuildersSelected11:52
LeanSparse Cuts GPU Memory for Sparse-Attention LLM Serving

The key insight is that prefill memory spikes, not just KV cache size, throttle concurrency — streaming layer histories through reusable buffers removes that admission bottleneck. Engineers running long-context inference on SGLang can adopt the design to serve more users per GPU without changing model weights. Compared with HiSparse and ECHO, the gains come from better indexing rather than more hardware.

arXiv: DeepSeek@Zaifeng Pan et al. (6)Source

Only the latest 2000 items are listed; for older ones see the daily archive