vLLM Matches DeepInfer Decode Performance on DeepSeek-V4.1-Flash
Open-source inference stacks closing the gap with a lab's proprietary serving setup changes the economics of self-hosting large MoE models. Teams weighing DeepSeek's API against running their own GPUs can use these Wide-EP and EPLB results to estimate revenue per GPU at off-peak prices. The GB200 and GB300 benchmarks also give infrastructure planners concrete throughput targets for the latest NVIDIA hardware.
The vLLM project reports that its Wide-EP and EPLB features can reproduce DeepSeek's serving performance. Under DeepSeek's decode configuration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 stays within 6% of DeepInfer's published throughput across batch sizes 8–256. On the AgentX workload with GB300, vLLM reaches up to 1M tokens/s/GPU and generates US$38.96/gpu/hr in revenue at 196 P90 TPS using DeepSeek's off-peak API pricing. New low-latency kernel support for DeepSeek-V4.1-Flash achieves up to 826 TPS per user on AgentX.
🚨🚨🚨 vLLM is all you need to reproduce DeepSeek’s performance.
With Wide-EP and EPLB, vLLM can match DeepInfer’s decode performance within 6% and generate $39/gpu/hr in revenue at DeepSeek’s off-peak pricing, at 200 P90 TPS on the AgentX workload.
Specifically: • Under DeepSeek’s decode configuration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 achieves throughput within 6% of DeepInfer’s published results across batch sizes 8–256. • On AgentX, vLLM pushes the Pareto frontier further on GB300, reaching up to 1M tokens/s/GPU and generating $38.96/gpu/hr in revenue at 196 P90 TPS, based on DeepSeek’s off-peak API pricing. • Also, vLLM extended low-latency kernel support for DeepSeek-V4.1-Flash, achieving up to 826 TPS per user on AgentX.
