NVIDIA开源了DFlash，用块扩散投机解码让Blackwell推理提速15倍，还支持SGLang和vLLM，随手就能用。
NVIDIA发布DFlash，一种开源轻量级块扩散模型，专为投机解码设计。在NVIDIA Blackwell硬件上，DFlash可实现高达15倍的推理吞吐量提升，同时保持相同的用户交互响应速度。与传统逐token解码不同，DFlash一次生成整个token块，由主模型并行验证。该方案即插即用，已集成到SGLang、TensorRT-LLM和vLLM等框架中。
Increase inference performance by up to 15x without sacrificing responsiveness. DFlash, an open so...
Increase inference performance by up to 15x without sacrificing responsiveness. DFlash, an open source lightweight block diffusion model designed for speculative decoding, delivers up to 15x higher throughput on NVIDIA Blackwell while maintaining the same user interactivity target. Instead of drafting tokens one at a time, it proposes a whole block in a single pass for the main model to verify in parallel. Adoption is drop-in with support in @lmsysorg SGLang, TensorRT-LLM, and @vllm_project . 💬 4 🔄 14 ❤️ 101 👀 8545 📊 23 ⚡