FreqDepthKV在长上下文推理中将KV缓存压缩3.9倍，同时保持任务精度，比此前压缩方法更稳定，适合内存受限场景。
FreqDepthKV是一种推理时KV缓存压缩方法，通过将相邻层的KV状态分解为共享低频深度分量和稀疏高频残差，并利用轻量在线探测根据注意力头对重建的贡献分配缓存模式。在32k token预填窗口下，FreqDepthKV在长上下文问答、检索、摘要和代码生成任务上达到58.3 Exact Match、63.0 F1、32.5 ROUGE-L和48.1 pass@1，接近全KV缓存水平。它还将解码吞吐提升至70.4 tokens/s，首令牌延迟降至2.06秒，峰值KV内存降至6.2 GB，实现3.9倍有效压缩比。
FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference
Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning. We introduce FreqDepthKV, an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals. A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits, allowing the compression policy to adapt to prompt structure without retraining. Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets. With a 32k-token prefill window, FreqDepthKV reaches 58.3 Exact Match, 63.0 F1, 32.5 ROUGE-L, and 48.1 pass@1, closely matching full KV while outperforming prior compressed-cache methods. It also improves decoding throughput to 70.4 tokens/s, reduces TTFT to 2.06 seconds, and lowers peak KV memory to 6.2 GB, achieving a 3.9x effective compression ratio.