SGLang给DeepSeek模型加了两个新负载均衡器，跑DeepSeek V3/R1速度能快最多7%，而且不改精度，想加速推理的可以试试。
SGLang团队为DeepEP MoE引入两种调度时负载均衡器Waterfill和LPLB。Waterfill将共享专家工作分配到较轻的rank，在DeepSeek V3/R1上带来+1.48%到+4.66%的性能提升，V4 Flash吞吐量从49,253 tok/s增至51,677 tok/s。LPLB优化冗余路由专家副本的流量分配，在red16/red32配置下取得+0.84%到+7.34%的提升。两种方法均不改变模型语义，保持推理精度。
🚀 New blog: Improving DeepEP MoE Load Balance in S…
🚀 New blog: Improving DeepEP MoE Load Balance in SGLang with Waterfill and LPLB
We're introducing two dispatch-time load balancers for DeepEP MoE. Even with EPLB, a single batch still hits ranks unevenly. Waterfill and LPLB fix that residual imbalance at runtime, no change to model semantics.
1️⃣ Waterfill for the dense shared expert Pours shared-expert work onto lighter ranks (“filling the valleys”) instead of every rank paying it locally. Near-zero overhead via shared-expert fusion into the DeepEP layout. ⚡️ +1.48% to +4.66% on DeepSeek V3/R1 across MMLU, GPQA, GSM8K ⚡️ V4 Flash: 49,253 → 51,677 tok/s (+4.92%)
2️⃣ LPLB for redundant routed-expert replicas EPLB splits hot experts evenly, but live traffic drifts from calibration. LPLB solves a per-layer min-max LP on-GPU each batch to split replica traffic optimally and shrink the busiest rank. ⚡️ +0.84% to +7.34%, strongest when redundant replicas exist (red16/red32)
Both preserve accuracy: same logical top-k, identical replica weights, only the physical rank changes.
Huge thanks to the @nvidia team for the collaboration!