想省显存又怕掉精度？这篇论文用通道级剪枝加4-bit量化，把MoE模型体积砍到1/5还能保住性能，DeepSeek和Qwen都能用。
该论文针对混合专家(MoE)模型部署时内存和推理开销大的问题，提出一种结构剪枝框架。方法将剪枝比率分配转化为通道分数覆盖最大化问题，通过归因近似高效求解。在DeepSeek和Qwen MoE模型上实验，结合4-bit量化后，50%或25%结构化剪枝仍保持模型准确率。在Qwen3-30B-A3B上，内存占用减少5.27倍，优于现有基线。
Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression
Mixture-of-Experts (MoE) models scale compute efficiently, yet remain expensive to deploy due to their substantial memory footprint and inference overhead. Prior compression methods mainly operate at the expert level, either removing entire experts or ranking experts by coarse-grained importance scores. However, such expert-wise decisions are often too coarse to capture fine-grained redundancy, leading to misallocated pruning budgets and limited compression. To address this problem, we observe that information within MoE experts is highly concentrated in a small subset of channels, leaving substantial redundancy even in experts deemed important. Based on this observation, we propose a structural pruning framework tailored for MoE models. Our method reformulates prune-ratio allocation as a channel-score coverage maximization problem and solves it efficiently using an attribution-based approximation. Experiments on DeepSeek and Qwen MoE models show that our method preserves model accuracy under 50% or 25% structured pruning when combined with 4-bit quantization. On Qwen3-30B-A3B, our approach reduces memory footprint by 5.27$\times$ and consistently outperforms state-of-the-art baselines across diverse benchmarks.