AITOPAI热报
Industry

vLLM 团队亮相阿里云 Apsara Conference 2026，谈开源推理与 Agent 工作负载

官方账号vLLM@vllm_project
Post
View the post
Why we picked it

vLLM 的人跑去阿里云大会分享推理经验，讲的是 Agent 时代开源推理怎么演进，做推理部署的可以看看

vLLM 项目成员 Kaichao You 在阿里云 Apsara Conference 2026 上分享了对开源推理的展望。

Body

他表示随着 AI 工作负载越来越 agentic，开源推理栈有许多新问题要解决。vLLM 与阿里云在开源推理方面保持合作。

Original · vLLM

Open-source AI is moving fast. There’s a lot to unpack as workloads become increasingly agentic.

Great to have @KaichaoYou share his perspective at #ApsaraConference2026.

Thanks @alibaba_cloud for having us. Excited to keep pushing open-source inference forward together. 🚀

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Industry" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include vLLM, 阿里云, ApsaraConference2026, 开源模型.

Source attribution: curated from vLLM (@vllm_project).

#vLLM#阿里云#ApsaraConference2026#开源模型#推理

Keep reading

Recent stories with the same tag or section
PapersBuildersMulti-sourceSelected12:56
RidgeFill Retrofit Speeds Prefill on Qwen and Llama Models

The non-obvious point is that YOCO-style self-forecasting caches, previously requiring dedicated training like DeepSeek-V4.1-Flash, can now be retrofitted onto off-the-shelf checkpoints with no training and no dropped tokens. Engineers deploying long-context inference in llama.cpp or vLLM can use it to cut time to first token on a single A100. Benchmarks like RULER and LongBench v2 let readers judge the quality cost themselves.

#RidgeFill#Qwen3.8#Llama-3.1#prefill optimization
Event
arXiv: DeepSeek@Dylan Johnston, Robert P. Smith5 sourcesSource
ModelsUS vs ChinaMulti-sourceSelected04:38
DeepSeek V4.1 Flash gains 1.5x speed on GB300 NVL72

Serving-stack software, not model changes, drove most of the speedup — a signal that inference infrastructure tuning is now as decisive as raw hardware for agentic workloads. Teams benchmarking DeepSeek deployments on GB300 or comparing against NVIDIA's Dynamo + vLLM reference setup can use AA-AgentPerf numbers as a realistic baseline for agentic coding performance.

#DeepSeek#NVIDIA GB300#vLLM#Dynamo
Event
官方账号Artificial Analysis@ArtificialAnlys11 sourcesSource
ProductsBuildersSelected00:03
vLLM Semantic Router team releases System Duo open-source routing framework

Routing cheap calls to small models and escalating only hard prompts can cut inference cost and latency without swapping out your backend stack. Builders running self-hosted LLM services can plug System Duo into an existing vLLM deployment and experiment with tiered serving. Compared with single-model serving, it shifts architecture decisions from model choice to traffic routing policy.

#vLLM#System Duo#Semantic Router#open-source models
官方账号vLLM@vllm_projectSource
ProductsBuildersMulti-sourceSelected03:32
vLLM Ecosystem Powers Kimi K3 and GLM-5.2 on TPUs

TPU inference for Chinese open-weight models signals that serving stacks are diversifying beyond NVIDIA, with vLLM TorchTPU as the concrete path to run models like GLM-5.2 and Qwen3.5 397B on Google silicon. Teams evaluating inference cost and hardware options should follow the SemiAnalysis write-up for benchmark details on the AgentX results.

#vLLM#TPU#Kimi K3#GLM-5.2
Event
官方账号vLLM@vllm_project6 sourcesSource