部署MoE模型的团队终于可以省下一半专家计算——ZEDA让Qwen3和GLM等模型自动跳过简单token，推理速度提升20%且几乎不掉精度，做模型推理优化的开发者可以直接参考论文方法。
研究发现，大型混合专家（MoE）模型在处理许多简单token时，浪费了约一半的专家计算资源。新提出的ZEDA（零专家自蒸馏适应）框架，通过为路由器添加“零专家”选项，让模型在token不需要复杂处理时直接跳过专家计算。该方法无需重新训练，而是将原MoE模型作为冻结教师，通过自蒸馏学习何时安全跳过计算。在Qwen3-30B-A3B和GLM-4.7-Flash上测试，去除了约50%的专家计算，精度损失极小，实际推理速度提升约20%。这表明计算消耗并不简单跟随任务难度，而是与不确定性相关，为部署MoE模型提供了更经济的方案。
A large MoE model may be wasting half its expert c…
A large MoE model may be wasting half its expert compute on tokens that barely need expert help.
In this paper 50% of expert computation removed, with almost no loss in accuracy.
This makes already-trained MoE models like Qwen3 and GLM stop calling half their experts when a token is too easy to need them.
Zero-Expert Self-Distillation Adaptation (ZEDA), a low-cost framework that transforms post-trained static MoE models into efficient dynamic ones.
Shows that many MoE tokens do not need real experts, only permission to skip them.
That sounds like a small routing trick, but it changes the economics of deployed language models.
Standard MoE models already avoid using every parameter, yet they still spend the same expert budget on every token.
ZEDA adds a strange new option to the router: experts that output exactly nothing.
When the model routes a token to one of these zero experts, it is not making the model dumber; it is admitting that this token does not need another expensive transformation.
The clever part is not the dummy expert, but the adaptation method.
Instead of retraining the model from scratch, the original MoE becomes a frozen teacher, while the new dynamic version learns when it can safely skip work.
Across Qwen3-30B-A3B and GLM-4.7-Flash, the result is roughly half the expert computation removed, with only marginal average accuracy loss and about 20% real inference speedup.
The deeper finding is: compute use did not simply track task difficulty.
The model spent more expert budget where uncertainty or teacher-student disagreement rose, while structured code and math fragments often needed less.
That makes ZEDA feel less like pruning and more like attention to computational doubt.
----
Paper Link – arxiv. org/abs/2605.18643
Paper Title: "Post-Trained MoE Can Skip Half Experts via Self-Distillation"