Papers

AI papers from arXiv and journals, each with a summary and why it matters · 7705 papers
Oct 30
PapersBuildersSelected10:41
FlatCake: Flat KVCache Storage for Faster LLM Serving

The key insight is that KV blocks are write-once and immutable, so caching hierarchies designed for general workloads actively hurt LLM serving — placing blocks on their final medium at creation outperforms migration-based designs. Engineers building or tuning inference infrastructure should evaluate flat layouts and GPUDirect Storage paths against Mooncake-style tiers, especially for agentic sessions with multi-gigabyte prefixes. The benchmark comparisons against Mooncake and Mooncake-GDS provide concrete baselines for such evaluations.

arXiv: DeepSeek@Xin Huang et al. (10)Source