CacheGraft 论文提出用 KV 缓存融合增强 RAG
CacheGraft: Enhancing Retrieval-Augmented Generation via Latent-Space Document Enrichment
一篇 RAG 新玩法论文：把文档离线灌成 KV 缓存，0.6B 小模型查询时直接吃缓存，30 组对比赢了 29 组，省算力场景挺实用。
论文 CacheGraft 基于 Cache-to-Cache（C2C）融合框架，把语料段落离线编码成 KV 缓存，在 4B 参数的 sharer 模型与 0.6B 参数的 receiver 模型之间做隐空间融合。查询时只需将缓存注入小模型，无需运行大模型。在 5 个 QA 基准、2 个本地评测模型下，CacheGraft 在 30 组对比中 29 组优于纯文本 baseline，其中 17 组统计显著。融合器零样本工作，从未用检索数据训练。
CacheGraft: Enhancing Retrieval-Augmented Generation via Latent-Space Document Enrichment
Recent work in machine learning has demonstrated that transformer-based language model agents can communicate state not only through text, but also directly in latent space via key-value (KV) caches. In this paper, we argue that such cache-to-cache translation can provide utility beyond the original motivating application of real-time agent communication and can instead be utilized for retrieval-augmented generation (RAG). Building on the Cache-to-Cache (C2C) fusion framework, our proposed system CacheGraft encodes each corpus passage offline into a fused KV cache representation between a larger sharer model (4B parameters in our experiments) and a smaller receiver model (0.6B parameters) such that this cache can be injected into the receiver at query time with no need for the larger model. Across five QA benchmarks and two independent locally-run judges, CacheGraft improves answer quality over the text-only receiver baseline in 29 out of 30 comparisons, of which 17 of the 30 are statistically significant. Notably, these gains are zero-shot with respect to the fuser, which was never trained on retrieval data. These initial results suggest that latent-space document enrichment may be a promising avenue for enhancing the performance of RAG systems, particularly in compute-constrained settings where it is only feasible to deploy a small model at inference time.