智谱的GLM-5.2开放权重模型,用DeepSeek V3.2的注意力机制加上自己的IndexShare,把1M长上下文推理搞便宜了,值得看看。
GLM-5.2是智谱发布的最新开放权重模型,基于GLM-5和GLM-5.1架构,复用了DeepSeek V3.2的Multi-head Latent Attention(MLA)和DeepSeek Sparse Attention(DSA)机制。新增的IndexShare机制在每四层运行一次完整索引器,后续三层复用选中的token索引,使100万token推理成本大幅降低。目前GLM-5.2在开放权重模型中表现最佳。
Just caught up with the recent GLM-5.2 release. Th…
Just caught up with the recent GLM-5.2 release. The best open-weight model today.
Architecture-wise, it's build on the GLM-5 and GLM-5.1 architecture that I covered previously, which means it's reusing the Multi-head Latent Attention (MLA) and DeepSeek Sparse Attention (DSA) mechanisms from DeepSeek V3.2. (I wrote about it here: https://t.co/tuunazfQ8y)
What's new is that they added an IndexShare mechanism. (That's a cross-layer reuse trick for DSA where instead of recomputing the sparse-attention top-k indexer in every layer, GLM-5.2 runs the full indexer only once every four layers and lets the following layers reuse those selected token indices. This keeps the same DSA idea but makes 1M-token inference much cheaper.)