ModelsSelected10月25日 02:02

vLLM Matches DeepInfer on DeepSeek-V4.1-Flash Decode

官方账号vLLM@vllm_project

Post

Why we picked it

An open-source inference stack catching up with a frontier lab's in-house engine means teams can serve DeepSeek-V4.1-Flash without relying on the vendor's own tooling. Engineers running self-hosted deployments in Singapore or ASEAN data centres can benchmark vLLM at DEP32 as a direct alternative to DeepInfer's EP64DP64 setup, avoiding engine lock-in while keeping comparable throughput at large batch sizes.

The vLLM project reports that it matches DeepInfer, DeepSeek's own serving engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration. The results hold across batch sizes 8 to 256, tested at DEP32 against DeepInfer's EP64DP64. DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and optimising prefill and decode, vLLM closed the performance gap.

Original · vLLM

vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64.

DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now matches DeepInfer's performance curves.

#vLLM #DeepSeek #DeepInfer #inference

Keep reading

Recent stories with the same tag or section

Models BuildersSelected11:25

MiniMax M3.1 Flash Runs in Production on vLLM

Production-grade serving of a MiniMax model through vLLM signals that the open-source inference stack can now carry commercial workloads with SLA guarantees. Teams evaluating cheaper open-weight serving options can look at Inferact's vLLM-based offering as a reference setup. Upstream contributions also mean the wider vLLM community inherits the reliability hardening that came out of this deployment.

#MiniMax #vLLM #open-source models #inference

官方账号vLLM@vllm_projectSource

Models Builders10:07

Chinese contentMiniMax M3.1 Flash 发布，vLLM Day-0 支持

MiniMax 新出的 M3.1 Flash，主打代码库工程和多步自动化，vLLM 发布当天就能跑，还带 NVFP4 量化和投机解码，自部署的可以试试。

#MiniMax #vLLM #开源模型 #推理模型

官方账号vLLM@vllm_projectSource

Products Builders09:33

Chinese contentMiniMax 与 Inferact 合作优化生产推理，改进回馈 vLLM

MiniMax 的 M3.1 Flash 交给了 Inferact 做生产推理，还把优化回馈给 vLLM，用 vLLM 的可以留意

#MiniMax #Inferact #vLLM #M3.1 Flash

MiniMax_AI@MiniMax_AISource

Products BuildersMulti-source09:08

Chinese contentGoogle 开源 TorchTPU，让 PyTorch 原生跑在 TPU 芯片上

Google 和 Meta 一起做了 TorchTPU，PyTorch 代码不用改就能跑 TPU，vLLM、SGLang 这些库都能配合，已开源在 GitHub。

#TorchTPU #PyTorch #TPU #vLLM

Event

Blognone@mk3 sourcesSource

Products BuildersOfficial02:28

Chinese contentLiquid AI 发布 RIPL 推理引擎，LFM 延迟降至 50 ms 以下

Liquid AI 推出 RIPL 推理引擎，官方称 LFM2.5-1.2B 的 p95 延迟比 vLLM 跑 Qwen3.5-0.8B 快十倍以上，适合做低延迟应用。

#Liquid AI #RIPL #LFM #vLLM

官方一手marktechpost@Asif RazzaqSource

Models BuildersSelected17:26

vLLM Ships Day-0 Support for Sage-A1.5

Day-0 support means the model can be self-hosted on existing vLLM infrastructure immediately, without waiting for community integrations. With only 3B active parameters, it targets teams running agentic workloads on modest local hardware who want an alternative to larger dense models. Developers comparing open-source MoE options for on-premise serving should evaluate its throughput against Qwen and DeepSeek MoE checkpoints.

#Sage-A1.5 #vLLM #Shanghai AI Lab #open-source models

官方账号vLLM@vllm_projectSource