这篇论文把MoE路由分歧讲透了，用分解和实验证明路由项影响很小，做自蒸馏的值得看看。
该论文研究同权重MoE自蒸馏中，教师和学生模型共享权重但路由不同专家的问题。通过精确的块级分解，将路由项与内容项分离，在7个开源检查点和2个领域上测试。结果显示路由项对块输出的影响仅1.6倍，残差流暴露为3.2倍。PubMedQA预注册实验表明，路由项移动输出的效果不到自然上下文效应的一半，且可被匹配范数噪声复现。结论是路由移动本身不能证明行为影响，需先测量暴露度。
Routing Divergence Is Not Evidence of Behavioral Influence in Same-Weight MoE Self-Distillation
Two Mixture-of-Experts (MoE) forward passes can share every weight yet route the same token through different experts. This creates a possible blind spot in same-weight self-distillation, where a demonstration-conditioned teacher supervises a query-only student. We study this mismatch in its single-step form, with frozen weights rather than as a proxy for a full training trajectory. An exact blockwise decomposition separates a routing term, which changes gates at fixed content, from a dense-like content term. Across seven open-weight checkpoints and two domains, the routing term spans only $1.6\times$ as a fraction of block output, while its residual-stream exposure spans $3.2\times$. Exposure is ordered by the routed block's share of the residual. Scaling the always-on backbone in two confirmatory models moves exposure monotonically; common-mode controls support a mass-and-coherence mechanism rather than denominator dilution alone. Preregistered PubMedQA patches on three models show that the full routing term moves outputs by less than half the natural context effect and is largely reproduced by matched-norm noise, whereas the content term is strongly direction-specific. Scale and merged-expert probes show that the narrow block-level range is not universal, although exposure remains small at the tested boundaries. Router movement alone is therefore not evidence of behavioral influence: measure exposure first, and use a behavioral intervention when the decision matters.