这篇论文把MoE剪枝的各种评分方法统一了，还提出MAN和MSAN两个新标准，在多个模型和基准上表现更稳定，适合做模型压缩的人参考。
该研究提出MoE专家一次性剪枝的统一公式，将现有启发式标准归为路由频率、门控权重、激活强度三类因素。基于此给出选择原则：任务无关剪枝应优先使用基于激活强度、无门控的标准。新提出的MAN和MSAN标准在4个MoE模型、16个基准上取得任务无关设置平均排名前两位。平均性能比最强基线提升最多8.8个百分点。
How to Score Experts for One-Shot MoE Expert Pruning: A Unified Formulation and Selection Principle
Mixture-of-Experts (MoE) language models reduce per-token computation through sparse expert activation, yet deployment still requires storing the full expert pool, making one-shot expert pruning a practical approach for reducing memory usage. Although effective, existing criteria are largely heuristic, and no single criterion is universally optimal. Thus, establishing a principle for selecting pruning criteria suited to different deployment objectives remains an important yet largely underexplored problem in one-shot expert pruning. To this end, we introduce a unified formulation for one-shot MoE expert pruning organized around three factors: routing frequency, gate weighting, and activation strength. The formulation yields a criteria selection principle: task-agnostic pruning should favor routed-token-averaged, gate-free activation-based criteria, whereas task-specific pruning can benefit from retaining routing-frequency and gate-weight information. Beyond this principle, the formulation also provides a systematic view of existing heuristic criteria and gives rise to two new task-agnostic criteria, Mean Activation Norm (MAN) and Mean Squared Activation Norm (MSAN). Across four representative MoE models and 16 diverse benchmarks, MAN and MSAN are consistently strong in the task-agnostic setting, obtain the top-two average ranks, and improve average performance by up to 8.8 points over the strongest baseline.