论文提出了DA-MoE，根据路由分布动态选最优内核，在DeepSeek-V3和Kimi K2上延迟最高降低56%，做MoE推理的同学可以关注这个优化思路。
混合专家（MoE）推理中专家GEMM的形状随运行时路由分布变化，现有系统使用静态令牌数桶选择融合内核，忽略每专家路由分布。论文提出分布感知框架，结合Effective Experts指标与Dirichlet逆向建模生成可控路由分布，并开发DA-MoE运行时系统，可在GPU上根据路由直方图与离线调优分布匹配选择近最优融合内核，无需CPU-GPU同步。在HumanEval-X服务轨迹上，DA-MoE在DeepSeek-V3和Kimi K2上将几何平均融合MoE延迟分别提升1.16倍和1.29倍，峰值加速达1.40倍和1.56倍。
Decoding the Skew: Distribution-Aware MoE Inference with Adaptive Kernel Dispatch
Mixture-of-Experts (MoE) inference consists of sparse expert GEMMs whose shapes vary with the runtime routing distribution. Existing serving systems typically select fused-MoE kernels using static token-count buckets, ignoring the per-expert routing distribution that determines tile padding, memory reuse, and kernel efficiency. We introduce a distribution-aware framework for modeling and benchmarking MoE inference. The framework combines the compact Effective Experts metric with a Dirichlet-based reverse-modeling procedure that generates controllable routing distributions for systematic hardware studies. Using it, we show that the best fused-MoE kernel changes with routing skew and token count. We further present DA-MoE, a GPU-resident kernel-dispatch runtime for NVIDIA GPUs that matches the live routing histogram to offline-tuned distributions and selects a near-optimal fused-MoE kernel without CPU--GPU synchronization. On HumanEval-X serving traces, DA-MoE improves geomean fused-MoE latency by 1.16X on DeepSeek-V3 and 1.29X on Kimi K2, with peak speedups of 1.40X and 1.56X.