这篇论文提出CAP，直接在专家放置和剪枝中考虑通信开销，实测比DeepSeek EPLB快1.2-1.8倍，适合跑大MoE模型的人。
CAP框架通过协同激活驱动的专家放置减少跨设备通信，并引入通信感知剪枝选择性移除路由目标。在单节点和多节点实验中，相比DeepSeek EPLB和vLLM的序列放置，吞吐量提升1.23倍至1.86倍。该方法在达到相同加速目标时能保持更好的模型准确率。
Communication-Aware Placement and Pruning for Efficient Mixture-of-Experts Inference
As MoE models scale to hundreds of experts, placement and pruning decisions increasingly dictate communication volume, affecting the performance of distributed inference across GPUs and nodes. We propose CAP (Communication-Aware Assignment and Pruning), a framework that considers computation, communication and accuracy together for efficient MoE inference through expert placement and pruning. It consists of three components: (1) Co-activation driven expert placement, which groups frequently co-activated experts to reduce inter-device and inter-node communication; (2) Communicationcomputation trade-off adjustment, which generates placements with different computational load and communication volume; and (3) Communication-aware expert pruning, which selectively removes routing destinations to reduce communication with limited accuracy degradation. By combining these components, CAP selects an efficient operating strategy for different hardware configurations. Across our single-node and multi-node experiments, it achieves 1.23x - 1.86 x throughput improvement over DeepSeek EPLB and sequential placement in vLLM, and preserves better model accuracy at the same target speedup under lossy acceleration.