阿里和蚂蚁团队搞了个优化，Qwen3-Omni实时对话延迟从6秒降到0.6秒，吞吐还翻了5倍多，推荐看技术博客。
Qwen3-Omni采用多模态Thinker与Talker（Code2Wav）流水线架构。高并发下仅复制语音阶段，复用Thinker结果，首音频延迟从约6秒降至0.6秒。吞吐量在同GPU上提升5.4倍，语音生成快于实时。该优化由阿里、蚂蚁集团SCT团队和vLLM-Omni团队共同实现。
🎙️ @Alibaba_Qwen's Qwen3-Omni listens, reasons, an…
🎙️ @Alibaba_Qwen's Qwen3-Omni listens, reasons, and talks back. Serving that in real time is a pipeline problem, not a single model: a multimodal Thinker, then Talker → Code2Wav for the speech.
Each stage bottlenecks differently, so the wins come from optimizing them layer by layer. One neat trick: under load, replicate only the two speech stages and let the heavy multimodal Thinker run once. At high concurrency that lands first audio in ~0.6s instead of ~6s, speech faster than real time, and ~5.4x the throughput on the same GPUs.
Built with @AntGroup's Super Computing Technology (SCT) team and the vLLM-Omni team. The blog breaks down the full stack, one bottleneck at a time 👇 🔗 https://t.co/aKWZdhNzMU