vLLM 和 NVIDIA 合作推出 DFlash 投机解码，Gemma-4 31B 推理速度提升近 6 倍，配置只需改一行 checkpoint 路径。
vLLM 项目宣布支持 DFlash 投机解码，用户只需将 EAGLE-3 检查点替换为 DFlash 检查点即可启用，无需修改代码。该功能通过开源 Speculators 库将 DFlash 草案模型与目标模型的隐藏状态连接。在单块 Blackwell Ultra GPU 上运行 Gemma-4 31B 模型，Math500 基准取得 5.8 倍吞吐量提升，GSM8K 提升 5.3 倍，HumanEval 提升 5.6 倍，MBPP 提升 4.4 倍。
🙏 Thanks to the @NVIDIAAI team for highlighting DF…
🙏 Thanks to the @NVIDIAAI team for highlighting DFlash support on vLLM!
With DFlash speculative decoding, swapping EAGLE-3 for a DFlash checkpoint is a config-only change — no code edits needed.
It runs through the open-source Speculators library, which links the DFlash drafter to the target model's hidden states in the vLLM inference path.
On Gemma-4 31B on a single Blackwell Ultra GPU, this delivers up to 5.8x higher throughput at the same concurrency over autoregressive decoding: 🧮 Math500 — 5.8x ➕ GSM8K — 5.3x 💻 HumanEval — 5.6x 🐍 MBPP — 4.4x
Read the blog here! 👇