论文提出DraftExpert，通过训练轻量草稿专家和预取机制，让端侧MoE模型推理速度提升1.45倍，适合关注边缘部署和推理加速的朋友。
DraftExpert针对在端侧设备（CPU-GPU或Flash-NPU卸载）部署大型MoE语言模型时的延迟问题，提出了一种扩展感知的自推测解码框架。它通过自蒸馏从冻结目标MoE中为每层训练一个轻量级驻留加速器的草稿专家，推理时使用固定足迹的共享+top-1+草稿专家解码器，结合置信度扩展截断和目标专家预取。在DeepSeek-V2-Lite和Moonlight-16B-A3B上，解码吞吐量平均提升1.45倍，草稿接受率达到84%~87%，预取命中率86%~88%。
DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference
Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory. We target latency-critical single-user settings where routed experts are staged on demand from CPU memory to a GPU or from Flash to a mobile NPU. In this setting, self-speculative decoding faces a new bottleneck: increasing the draft expert set improves accuracy but triggers extra expert loading, while cheap small-footprint drafts have low acceptance; moreover, verifying a multi-token block activates the union of target experts and is no longer close to one target step. We propose DraftExpert, an expansion-aware self-speculative decoding framework for expert-offloaded MoE inference. DraftExpert trains one lightweight accelerator-resident draft expert per layer by self-distilling residual, logit/token, and router-agreement signals from the frozen target MoE. At inference time, it uses a fixed-footprint shared+top-1+draft-expert drafter together with confidence--expansion truncation and target-expert prefetching, while final tokens are still exactly verified by the target model. On DeepSeek-V2-Lite and Moonlight-16B-A3B across CPU-GPU and Flash-NPU offload, DraftExpert improves decode throughput by 1.45x on average, raises draft acceptance to 84~87%, and achieves 86~88% prefetch hit rates.