KV缓存驱逐会掉精度，KV-Rescue用个小模型当辅助，能救回87%的损失，还省43%的生成量，跑长推理的可以看看。
KV-Rescue是一种无需训练的推理框架，通过轻量级全上下文助手桥接KV驱逐引入的信息缺口。在Qwen2.5-Math 7B和72B模型上，五个数学基准测试中，KV-Rescue在驱逐预算B=64时平均恢复了87%的精度损失。该框架使用在线检测器基于熵和可压缩性提前终止不连贯或重复的生成，将基础模型令牌生成量平均减少43%。
KV-Rescue: Recovering Reasoning Language Model KV Eviction Loss via Stepwise Interleaving
KV-cache eviction caps the memory cost of long reasoning traces but is inherently lossy because the model decodes from a partial view of its history. Under aggressive budgets, this not only lowers accuracy but can also cause runaway degeneration, where the model produces incoherent or repetitive tokens until reaching the length limit. We characterize much of this loss as an information gapf caused by missing context, rather than a capability gap caused by limited model capacity. An evicted 7B model and a full-context 1.5B model make complementary errors, and an oracle choice between their answers recovers 79% of the accuracy gap to the full-KV 7B model. Based on this observation, we propose KV-Rescue, a training-free inference framework that bridges the information gap introduced by KV eviction using a lightweight full-context helper. KV-Rescue interleaves reasoning steps from the two models into a shared trajectory. An online detector uses entropy and compressibility to terminate the generation of incoherent or repetitive base-model candidates early. Across five math benchmarks with Qwen2.5-Math 7B and 72B, KV-Rescue recovers an average of 87% of the accuracy lost to eviction at eviction budget B=64. A decode-cost analysis further shows that preventing runaway degeneration cuts base-model token generation by 43% on average.