这篇论文分析了Qwen、DeepSeek等MoE模型的专家加载瓶颈，用ST-MoE框架通过预取专家来加速推理，兼顾效率和精度，适合关注大模型推理优化的读者。
针对MoE模型（如Qwen、DeepSeek）推理中专家加载延迟高的问题，研究者分析了专家选择行为，发现相邻MoE层和连续解码token间专家请求存在强相关性。基于此提出ST-MoE，一个结合轻量级运行时预测机制与可重构硬件设计的专家预取框架。ST-MoE通过预取专家与计算重叠，显著提升推理性能并降低能耗，同时保持模型精度。实验在多种MoE模型和应用（语言理解、代码生成）上验证了有效性。
A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference
Mixture-of-Experts (MoE) based large language models (LLMs), such as Qwen and DeepSeek, have recently emerged as an effective approach to improving model capacity without proportionally increasing computational cost. By replacing the conventional feed-forward network in dense LLMs with a set of experts and activating only a subset of them for each input token, MoE models significantly increase the total number of parameters while keeping the per-token computation relatively manageable. However, this dynamic and irregular expert activation pattern also introduces substantial expert loading overhead during inference, since the required experts must be fetched on demand according to token-dependent routing results. As a result, expert loading latency becomes a major source of performance and energy inefficiency. To this end, we first perform a comprehensive analysis of expert selection behavior in various MoE-based LLMs and applications, including language understanding and code generation. Our analysis reveals that, within each application domain, expert requests exhibit strong correlation across both adjacent MoE layers and consecutive decoding tokens, making future expert activations predictable. Based on this insight, we propose ST-MoE, a spatio-temporal expert prefetching framework that proactively stages experts ahead of use to overlap expert loading with ongoing computation. ST-MoE combines a lightweight runtime prediction mechanism that preserves the original routing behavior with a reconfigurable hardware design that efficiently supports dynamic expert prefetching. The combined effect of the prediction mechanism with the supporting hardware significantly improves MoE inference performance and energy efficiency while preserving model inference accuracy.