这篇论文把MoE路由的几何重叠和功能价值分开了，用了新指标ESSI和受控实验，结论很扎实，比单纯算相似度靠谱。
本研究提出专家子空间分离指数（ESSI）和匹配路由残差等方法，区分路由一致性、候选质量和上下文交互。在六种MoE架构中发现专家子空间高度重叠，但实际路由对token表征的解释优于匹配替代方案。在OLMoE、Mixtral和DeepSeek的39个阶乘单元中，选中专家在每个单元都比最强的未选中候选解释更多残差，但实际前缀在所有交互中均缩小该优势，所有95%置信区间低于零。尽管几何重叠，添加后续专家在39个冻结路由比较中改善24个单元的下一token预测，其余15个不显著，受控训练中三个种子均偏好Top-2优于Top-1。这一模式表明路由从共享几何邻域选择token相关专家，而多专家计算无需不相交线性覆盖即可保持效用。
Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing
Sparse mixture-of-experts (MoE) language models route each token to multiple experts, suggesting a geometric account of their benefit: co-selected experts should contribute distinct representation directions. Existing evidence often conflates route coherence, candidate quality, and candidate-by-context interaction. We distinguish these quantities using an Expert Subspace Separation Index (ESSI), matched-route residuals, and a prefix-controlled $2\times2$ factorial; frozen-route interventions and a controlled Top-$k$ study assess functional value. Three paired contrasts organize the findings. First, across six MoE architectures, expert subspaces overlap substantially, yet actual routes explain token representations better than matched alternatives. Second, across the 39 factorial cells in OLMoE, Mixtral, and DeepSeek, the selected candidate explains more of the residual representation than the strongest unselected rival in every cell, yet the actual prefix narrows this advantage throughout: all interactions are negative, and every 95% confidence interval lies below zero. Third, this geometric narrowing does not imply functional redundancy: adding later experts improves next-token prediction in 24 of 39 frozen-route comparisons, while the other 15 estimates are inconclusive; a controlled training study also favors Top-2 over Top-1 in all three seeds. We call this joint pattern coherent overlap: routing selects token-relevant experts from a shared geometric neighborhood, while useful multi-expert computation persists without disjoint linear coverage. Separating these quantities clarifies why geometric similarity alone cannot determine redundancy or pruning value.