NVIDIA压缩MoE模型Puzzle-75B-A9B，吞吐量翻倍可保持质量
英伟达把大MoE模型压到9B活跃参数，吞吐量翻倍，智能体应用部署成本大幅下降，适合模型高效推理场景。
NVIDIA发布论文，将混合MoE模型Nemotron-3-Super压缩为Puzzle-75B-A9B，活跃参数降至9B。在单个8xB200节点上，交互式服务器吞吐量约为原模型2倍。在H100 GPU上，1M token并发数从1请求提升至8。在推理、编程、长上下文和智能体基准上精度保持。该方法联合优化异构MoE剪枝、活跃参数预算和Mamba剪枝，结合蒸馏、强化学习、量化与多令牌预测头。
Banger compression paper from NVIDIA. (bookmark it) Bigger MoE models keep winning on quality, but serving them at interactive latency is still hard. NVIDIA compresses the hybrid MoE Nemotron-3-Super into Puzzle-75B-A9B and roughly doubles interactive server throughput while holding quality. Pay attention to the joint structural search. Heterogeneous MoE pruning, active-parameter budget, and Mamba pruning get optimized together rather than one at a time, wrapped in an iterative pipeline with distillation, RL, quantization, and a Multi-Token Prediction head. Why does it matter? On a single 8xB200 node it hits about 2x the parent's server throughput at matched user-throughput, and 1M-token concurrency on a single H100 climbs from 1 request to 8. Accuracy holds across reasoning, coding, long-context, and agentic benchmarks. Cheaper serving with agentic capability intact changes what you can afford to run with these models. Paper: arxiv.org/abs/2607.04371 Learn to build effective AI agents in our academy: academy.dair.ai 💬 7 🔄 7 ❤️ 16 👀 3257 📊 12 ⚡
