DeepSeek改了Engram的哈希方式，不同tokenizer之间不用再各自训练，效果还一样，做多语言模型的可以看看。
DeepSeek的Engram条件记忆模块在存储与推理之间做权衡，但依赖token级N-gram哈希，导致不同tokenizer需从头训练。论文提出用多项式哈希替换XOR哈希，建立跨N的联合嵌入空间。实验显示该改动在保持性能的同时实现tokenizer无关性，字节等价序列哈希相等。
Tokenizer-Agnostic Engram Module
Deepseek's Engram, a conditional memory module, was introduced to trade-off storage versus reasoning in large language models. However, the module relies on token-level $N$-gram hashing for Engram embedding lookup, introducing a tight coupling to the tokenizer used: a model with a different tokenizer would have to train its own Engram embeddings from scratch. To improve the reusability of Engram embeddings, we propose a change to the hashing routine, enabling compatibility between Engram models using different tokenizers. Instead of modelling disjoint $N$-gram spaces, we treat $N$-gram as a method to sample potentially useful byte sequences, from all possible byte sequences across tokens. We replace the XOR-based hashing with the general polynomial hashing with a joint embedding space across $N$. This work investigates the possible trade-offs and shows that this simple substitution produces comparable performance and achieves tokenizer-agnosticism: hash equivalence for byte-equivalent token sequences.