GateRQ：按门控精度压缩 DeepSeek 条件记忆表，GPU 内存省 7.8 倍
Let the Gate Decide: Per-Lookup Precision for Quantized Conditional Memory in LLMs
DeepSeek 那张巨大的记忆表这次被压缩了 7.8 倍，vLLM 吞吐还涨了 11%-31%，PopQA 只掉 1 分，做推理部署的可以看看。
DeepSeek-V4.1-Flash 上的条件记忆层（如 Engram）移除后 PopQA 会掉 36.8 分，但这类哈希 n-gram 表体积大、难量化。论文发现按访问频率分配比特的常见做法反而不如随机分配，原因是检索行的量化误差被门控加权：高频 n-gram 门控小，长尾实体门控大。GateRQ 用 1-bit 残差码阶梯存储每行，只对门控排名最高的查找读取更深层级。无需重训练，GateRQ 在 DeepSeek-V4.1-Flash 上把 GPU 表内存压缩 7.8 倍，vLLM 吞吐比 FP8 表提升 11%-31%，每查找读 1.50 bit 时 PopQA 只掉约 1 分。
Let the Gate Decide: Per-Lookup Precision for Quantized Conditional Memory in LLMs
Conditional memory layers, such as DeepSeek's Engram, attach large hashed n-gram tables to a language model, and removing them from DeepSeek-V4.1-Flash costs 36.8 PopQA points. Low-bit quantization loses long-tail facts first, and the standard remedy, giving more bits to frequently accessed rows, backfires: refining lookups by access frequency is worse than refining them at random. We trace this to the gate that scales each retrieved row. A lookup's quantization error is weighted by its gate, which is small for frequent n-grams, large for the rare n-grams of long-tail entities, and computable from a 1-bit code. GateRQ therefore stores each row as a ladder of 1-bit residual codes, keeps level 1 in GPU memory to compute the gate, and reads deeper levels only for the lookups the gate ranks highest. Without retraining, GateRQ cuts GPU table memory by 7.8x on DeepSeek-V4.1-Flash, which raises vLLM throughput by 11%-31% over FP8 tables, while losing about 1 PopQA point at 1.50 bits read per lookup and 0.2 points at 2.10 bits across two models.
