Nemotron 3 的架构创新直击大模型推理效率瓶颈，做模型优化和部署的开发者值得关注其混合 Mamba 和潜在 MoE 的具体实现，可以直接参考其设计思路。
NVIDIA 发布 Nemotron 3 模型，采用混合 Mamba Transformer 架构，通过 Mamba-2 降低注意力机制开销，实现亚二次复杂度。潜在 MoE 通过降维投影减少 HBM 与 SRAM 间的数据移动，并增加专家数量以提升稀疏性效率。多 token 预测（MTP）使模型在训练时能预见未来 token，推理时可用于推测解码。模型采用新的 OpenMDW 1.1 许可证。
Shoutout to Caleb for putting together a great deep dive on Nemotron 3 🙌 Check it out.
Shoutout to Caleb for putting together a great deep dive on Nemotron 3 🙌 Check it out. Caleb Eom @calebfoundry Nemotron 3 Full Breakdown With the help of Joey Conway from @NVIDIAAI getting into the specifics around why Nemotron 3 is kind of a big deal Biggest headline with Nemotron is: Hybrid Mamba Transformer, Latent MoE, and MTP Hybrid Mamba Transformer essentially attacks right at the Attention mechanism to make the overhead sub-quadratic, but unlike quantizing KV Cache or swapping out attention head, NVIDIA chose Mamba-2 Latent MoE helps further optimize on sparsity by down projecting the dimensions so you're doing less math and less memory movement between HBM and SRAM, you're saving a ton, and NVIDIA made a conscious choice to add more experts given the surplus Finally, MTP or multi token prediction where the model can see future tokens to be more expressive in training and also option to use for speculative decoding during inference Oh, also the model adopts the new OpenMDW 1.1 License Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet 💬 5 🔄 1 ❤️ 16 👀 1221 📊 5 ⚡