做 MoE 系统优化或互联设计的工程师，这篇论文用实测数据推翻了行业两个核心假设，直接告诉你路由不均衡是模型固有、模拟测试不可信，看完能避免在错误方向上投入精力。
MoE 专家并行中的 AlltoAll 调度是主要瓶颈，业界提出四种缓解方案，但都基于两个假设：路由不均衡可由系统层修正，以及模拟 token 基准能代表真实生产路由。DODOCO 工具在五个 MoE 检查点（DeepSeek-V2-Lite MLA、DeepSeek-MoE-16B MHA、Qwen3-30B GQA、Nemotron-30B Mamba-2、Qwen3.5-35B GDN）上测试了这些假设，发现两者均不成立。扩展 EP 时，每个专家的最大/平均 token 比变化不超过 5%，表明 straggler 是模型路由决策固有的，而非专家在 rank 上的分布。模拟 token 高估路由 Gini 系数最多 2.35 倍，并制造出虚假的 batch-size 缩放趋势。五种架构分为两个稳定带：MHA 和 Mamba-2 在真实文本上 Gini 降至 0.105-0.150，而 MLA 和 GDN 始终高于 0.24。
Diagnosing Overhead in Dispatch Operations: Cross-architecture Observatory
AlltoAll dispatch is the dominant bottleneck of MoE expert parallelism, and the interconnect community has responded with four families of mitigations: predictive sample placement, adaptive expert relayout, hierarchical collectives, and EP-aware topology. All four rest on two assumptions about the workload. The first is that routing imbalance is correctable by the system layer. The second is that the mock-token benchmarks evaluating them faithfully represent production routing. We introduce DODOCO to test both assumptions. We instrument five MoE checkpoints spanning five sequence-mixer designs (DeepSeek-V2-Lite MLA, DeepSeek-MoE-16B MHA, Qwen3-30B GQA, Nemotron-30B Mamba-2, Qwen3.5-35B GDN) under a 5 by 6 grid of data conditions plus a matched EP scan from 4 to 32 ranks on H100s; both assumptions fail. Scaling EP changes the per-expert max/mean token ratio by at most 5% within every architecture's measurable range: the straggler is intrinsic to the routing decision the model makes, not to how its experts land on ranks. Mock tokens overestimate routing Gini by up to a factor of 2.35 and fabricate a batch-size scaling trend that vanishes the moment real text replaces random IDs. A third pattern, unexpected, emerges from the same matrix: the five architectures cleave into two stable bands. MHA and Mamba-2 (data-resilient) drop to Gini 0.105 and 0.150 on wikitext. MLA and GDN (persistently concentrated) stay above 0.24 on every real-text condition and reach 0.29 to 0.38 on mock. GQA is the intermediate case. These bands, not the EP degree or the mock-data profile, are the right workload input to AlltoAll-aware interconnect and dispatch design.