Netpreme的X-Mem内存插进SGLang的HiCache，多轮对话缓存带宽不再是瓶颈，TTFT降6.7倍，吞吐量涨30%，实测SWE-bench效果显著。
Netpreme发布博客，介绍将X-Mem MPU插入SGLang HiCache的分层KV缓存方案。在SWE-bench的Claude Code多轮编码中，前缀缓存命中率平均达98%。用X-Mem替换主机DRAM作为卸载层级后，TTFT降低6.7倍，每用户TPS提升33-50%，系统吞吐量提升30%。该方案通过HiCache的分层接口实现即插即用。
🚀 Accelerating SGLang HiCache with Netpreme X-Mem™…
🚀 Accelerating SGLang HiCache with Netpreme X-Mem™ MPU: new blog from @netpreme on plugging a TB/s memory tier into HiCache's hierarchical KV cache!
In multi-turn coding sessions (Claude Code on SWE-bench), prefix-cache hit rates average ~98%. At this point, KV bandwidth is the bottleneck, not compute. Swapping Host DRAM for X-Mem™ as the offload tier: ✅ Up to 6.7× lower TTFT ✅ 33–50% higher TPS/user under load ✅ 30% higher system throughput ✅ Drop-in via HiCache's tiering interface