英伟达把大MoE模型压到9B活跃参数，吞吐量翻倍，智能体应用部署成本大幅下降，适合模型高效推理场景。
NVIDIA发布论文，将混合MoE模型Nemotron-3-Super压缩为Puzzle-75B-A9B，活跃参数降至9B。在单个8xB200节点上，交互式服务器吞吐量约为原模型2倍。在H100 GPU上，1M token并发数从1请求提升至8。在推理、编程、长上下文和智能体基准上精度保持。该方法联合优化异构MoE剪枝、活跃参数预算和Mamba剪枝，结合蒸馏、强化学习、量化与多令牌预测头。
Banger compression paper from NVIDIA. (bookmark it) Bigger MoE models keep winning on quality, but...
Banger compression paper from NVIDIA. (bookmark it) Bigger MoE models keep winning on quality, but serving them at interactive latency is still hard. NVIDIA compresses the hybrid MoE Nemotron-3-Super into Puzzle-75B-A9B and roughly doubles interactive server throughput while holding quality. Pay attention to the joint structural search. Heterogeneous MoE pruning, active-parameter budget, and Mamba pruning get optimized together rather than one at a time, wrapped in an iterative pipeline with distillation, RL, quantization, and a Multi-Token Prediction head. Why does it matter? On a single 8xB200 node it hits about 2x the parent's server throughput at matched user-throughput, and 1M-token concurrency on a single H100 climbs from 1 request to 8. Accuracy holds across reasoning, coding, long-context, and agentic benchmarks. Cheaper serving with agentic capability intact changes what you can afford to run with these models. Paper: arxiv.org/abs/2607.04371 Learn to build effective AI agents in our academy: academy.dair.ai 💬 7 🔄 7 ❤️ 16 👀 3257 📊 12 ⚡