这篇论文用OLMoE和DeepSeek-V2-Lite实测证明，MoE路由的选专家和加权输出不该共用一套分数，FDAA方法能白捡性能提升，做MoE优化的值得看。
一项新研究质疑稀疏MoE路由器中专家选择与输出加权是否应使用相同分数。在预训练的OLMoE-1B-7B上，仅改变集合内聚合方式，结构化oracle将全周期交叉熵提升0.0160±0.0039。研究者训练了301K参数的FDAA后处理头，在冻结主干、路由器和专家的情况下，将WikiText-103测试集Delta CE提升至-0.1523±0.0031。在DeepSeek-V2-Lite上复现固定调度审计，路由器Top1识别最佳选中专家的比例仅为12.5%和16.7%。结果表明专家选择与专家承诺应跨架构区分。
Beyond Routing: Decoupling Expert Dispatch and Aggregation in Sparse Mixture-of-Experts
Sparse Mixture-of-Experts (MoE) routers commonly use the same scores both to select experts and to weight their already-computed outputs. We study whether these two roles, dispatch and aggregation, should be coupled. On pretrained OLMoE-1B-7B, we keep selected Top-8 expert IDs, expert computation, and total selected router mass fixed and change only within-set aggregation. A structured oracle improves full-horizon cross-entropy by 0.0160 +/- 0.0039 across three seeds; the router's top-scored expert is the counterfactual-best vertex only 17.2% of the time, with router-utility Spearman 0.030. We therefore train Fixed-Dispatch Adaptive Aggregation (FDAA), a 301K-parameter post-compute head optimized directly with the language-modeling objective while freezing the backbone, router, and experts. On OLMoE, FDAA improves fresh WikiText-103 test by Delta CE = -0.1523 +/- 0.0031 across three seeds, and mixed-domain training gives robust gains on WikiText-103, C4, and held-out Penn Treebank under frozen confirmatory evaluation. We also replicate the fixed-dispatch audit on DeepSeek-V2-Lite, which uses Top-6 routed experts plus shared experts. Best-vertex headroom remains significant on WikiText and C4, while router Top1 identifies the best selected expert in only 12.5% and 16.7% of audited examples. In a one-seed mixed-domain replication, FDAA improves locked WikiText and PTB, while C4 is statistically neutral. These results support a cross-architecture distinction between expert selection and expert commitment.