vExpert：面向分布式 MoE 推理的虚拟化专家存储方案
vExpert: Virtualizing Expert Storage for Adaptive Load Balancing in Distributed MoE Inference
分布式跑 MoE 模型经常被负载不均拖慢，这篇用虚拟化专家存储的思路，在 8 卡 H100 集群跑 DeepSeek-V3.1 提速 1.2 倍，做推理系统的可以看看。
论文提出 vExpert，将 DRAM 卸载范式从部分驻留改为全冗余的虚拟化专家存储。它建立了首个同时刻画 H2D 延迟、PCIe 争用与负载不均衡的性能模型，并配有轻量级分配器动态映射物理专家到虚拟槽位。针对多进程环境设计了分离式管理器来降低内存开销。在 8 节点 H100 集群上对 DeepSeek-V3.1 的评测中，负载均衡比提升最高 55%，端到端加速 1.2 倍。
vExpert: Virtualizing Expert Storage for Adaptive Load Balancing in Distributed MoE Inference
Mixture-of-Experts (MoE) models have become prevailing, yet their distributed deployment suffers from severe dynamic load imbalance due to the conflict between static expert placement and inherent routing dynamism. While host DRAM offers capacity to avoid static device storage for load balancing, we identify existing offloading works fundamentally fail in distributed inference due to two overlooked challenges: (1) coupled performance impacts of runtime Host-to-Device transfers due to load dynamism and shared PCIe topology; (2) system-level memory overhead incurred by DRAM usage at scale. In this paper, we propose vExpert that shifts DRAM offloading paradigm from partial residency to full redundancy through virtualized expert storage. We pioneer the first systematic performance model that jointly captures H2D latency, PCIe contention, and load imbalance, building a lightweight allocator to dynamically map physical experts to virtual slots runtime. A novel disaggregated manager is designed to reduce memory overhead in multi-process environments. Evaluated on DeepSeek-V3.1 within an 8-node H100 cluster, vExpert improves load balance ratio by up to 55% and achieves 1.2$\times$ end-to-end speedup.