TraceKV：免训练低秩 KV 压缩，5.64 倍压缩保推理精度
TraceKV: Policy-Preserving Low-Rank KV Cache Compression for Reasoning Vision-Language Models
推理型 VLM 的 KV 缓存太吃显存，这篇论文给出免训练方案：在线校准加敏感度 SVD，5.64 倍压缩基本不掉点，解码还能快 1.83 倍。做多模态部署、被长推理链显存困扰的团队值得看，代码说会开源。
TraceKV 是一个面向推理型视觉语言模型（VLM）的免训练低秩 KV 缓存压缩框架，来自 arXiv 论文 2611.15569。它针对现有低秩压缩在压缩推理 KV 状态时引发策略漂移的问题，找出两个根因：静态上下文与在线推理状态之间的校准失配，以及不同 KV 状态的错误敏感度不均。
方法上，TraceKV 在同一个策略保持原则下统一三项技术：在线策略校准（on-policy calibration）、敏感度感知 SVD 分解和全局秩分配，显著降低推理过程中的 policy-KL 散度。结合量化后可在 5.64 倍 KV 缓存压缩下保持近乎无损的推理准确率；优化后的低秩注意力核还带来最高 1.83 倍的端到端解码加速。
该工作针对的是推理链越来越长的多模态模型在显存和解码速度上的瓶颈，代码将开源在 GitHub（AK-DREAM/TraceKV），对部署推理型 VLM 的开发者可直接复用。
TraceKV: Policy-Preserving Low-Rank KV Cache Compression for Reasoning Vision-Language Models
Reasoning vision-language models (VLMs) impose substantial KV cache demands from both large multimodal contexts and growing reasoning traces. Low-rank KV compression has emerged as a promising paradigm that stores low-dimensional latents of KV states and can be combined with quantization. However, existing low-rank methods often induce growing predictive policy drift from the dense model when compressing reasoning KV states, resulting in reasoning failures. We identify two key causes of this degradation: calibration-state mismatch between static contexts and on-policy reasoning states, and unequal error sensitivity across KV states. To address these challenges, we propose TraceKV, a training-free low-rank KV compression framework that unifies on-policy calibration, sensitivity-aware SVD, and global rank allocation under a single policy-preserving principle, substantially reducing policy-KL divergence during reasoning. Experiments across multiple VLMs and visual reasoning benchmarks demonstrate superior accuracy--memory trade-offs, achieving near-lossless reasoning accuracy with 5.64$\times$ KV cache compression when combined with quantization. Optimized low-rank attention kernels further deliver up to 1.83$\times$ end-to-end decoding speedup. Code will be available at https://github.com/AK-DREAM/TraceKV.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include TraceKV, VLM, KV缓存压缩, 推理模型.
Source attribution: curated from arXiv cs.LG (@Ke Li, Zishan Shao, Yiran Chen).