Tencent paper shows MoE models survive neuron-level pruning
The practical takeaway is that expert granularity is the wrong unit for MoE compression — neuron-level pruning keeps skills intact while cutting memory by over a third. Teams running self-hosted MoE models on constrained GPU budgets, from inference startups to enterprise ML platforms, can apply RAMPr-style scoring to cut serving costs without retraining. The results give open-source MoE deployments a concrete alternative to expert-dropping methods.
