推理模型的长输出让内存和计算成本飙升，VaSE用随机淘汰策略在4倍压缩下保持高准确率，做推理模型优化的开发者可以直接参考论文实现。
推理模型通过长思维链提升准确性，但长输出导致内存和计算瓶颈。现有KV缓存淘汰方法在压缩缓存时会丢失关键信息，导致模型陷入重复推理循环。研究发现，少量值状态具有异常大的幅度，淘汰它们会引发灾难性失败；引入随机性可提高缓存多样性。基于此，研究者提出VaSE方法，无需训练即可保护大幅度值状态并促进多样化淘汰决策。在6个推理任务上，Qwen3模型使用VaSE实现4倍KV缓存压缩，准确率超过最强淘汰方法4%以上，弥合了效率与准确性之间的差距。
Value-Aware Stochastic KV Cache Eviction for Reasoning Models
Reasoning models improve accuracy through extended chains of thought, but their long outputs create a memory and compute bottleneck. KV cache eviction methods reduce this cost by evicting unimportant key-value pairs from the cache, yet they often yield worse accuracy than selection-based sparse attention alternatives, which keep the full KV cache. We identify key factors crucial to KV cache eviction accuracy. First, a small fraction of value states have abnormally large magnitudes, and evicting them causes catastrophic failure where models enter repetitive reasoning loops. Second, introducing stochasticity during eviction improves accuracy by increasing cache diversity. Based on these findings, we propose Value-aware Stochastic KV Cache Eviction (VaSE), a training-free recipe that protects large-magnitude value states and promotes diverse eviction decisions. Across six reasoning tasks, Qwen3 models using VaSE with 4x KV cache compression yield higher average accuracies than SOTA selection method at the same sparsity, while outperforming the strongest eviction method by more than 4%. Overall, VaSE bridges the gap between efficiency and accuracy, supporting FlashAttention2 and enabling a static memory footprint for reasoning models.