阿里Qwen3-Omni实时语音推理优化:首音延迟降至0.6秒
阿里和蚂蚁团队搞了个优化,Qwen3-Omni实时对话延迟从6秒降到0.6秒,吞吐还翻了5倍多,推荐看技术博客。
Qwen3-Omni采用多模态Thinker与Talker(Code2Wav)流水线架构。高并发下仅复制语音阶段,复用Thinker结果,首音频延迟从约6秒降至0.
6秒。吞吐量在同GPU上提升5.
4倍,语音生成快于实时。该优化由阿里、蚂蚁集团SCT团队和vLLM-Omni团队共同实现。
🎙️ @Alibaba_Qwen's Qwen3-Omni listens, reasons, and talks back. Serving that in real time is a pipeline problem, not a single model: a multimodal Thinker, then Talker → Code2Wav for the speech.
Each stage bottlenecks differently, so the wins come from optimizing them layer by layer. One neat trick: under load, replicate only the two speech stages and let the heavy multimodal Thinker run once. At high concurrency that lands first audio in ~0.6s instead of ~6s, speech faster than real time, and ~5.4x the throughput on the same GPUs.
Built with @AntGroup's Super Computing Technology (SCT) team and the vLLM-Omni team. The blog breaks down the full stack, one bottleneck at a time 👇 🔗 https://t.co/aKWZdhNzMU
This report is categorized under AITOP's "Models" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Qwen3-Omni, multimodal, inference optimization, voice conversations.
Source attribution: curated from vLLM (@vllm_project).