AITOPAI热报
ProductsSelected

vLLM Ecosystem Powers Kimi K3 and GLM-5.2 on TPUs

官方账号vLLM@vllm_project
Post
View the post
Why we picked it

TPU inference for Chinese open-weight models signals that serving stacks are diversifying beyond NVIDIA, with vLLM TorchTPU as the concrete path to run models like GLM-5.2 and Qwen3.5 397B on Google silicon. Teams evaluating inference cost and hardware options should follow the SemiAnalysis write-up for benchmark details on the AgentX results.

The vLLM project thanked SemiAnalysis for highlighting the ecosystem's work on TPU inference. The coverage covers the Kimi K3 megakernel and vLLM TorchTPU, which power AgentX results across GLM-5.2, Qwen3.5 397B, and Kimi K3. The project credited @inferact and Google Cloud for pushing vLLM performance on TPU.

Original · vLLM

Thanks to @SemiAnalysis_ for highlighting the vLLM ecosystem’s work on TPU, in particular the Kimi K3 megakernel and vLLM TorchTPU powering AgentX results across GLM-5.2, Qwen3.5 397B, and Kimi K3.

Shoutout to @inferact and @googlecloud for pushing vLLM performance on TPU. The @SemiAnalysis_ write-up below is a great read 👇

#vLLM#TPU#Kimi K3#GLM-5.2#open-source models

Keep reading

Recent stories with the same tag or section
IndustryBuildersSelected03:02
vLLM Ranks #2 in GitHub Octoverse Community Engagement

High community engagement on vLLM signals a broad, active contributor base across model and hardware support, which reduces the risk of relying on it for production inference stacks. Teams deploying LLM inference in ASEAN can weigh that maturity when choosing between vLLM and alternatives like SGLang or TensorRT-LLM.

#vLLM#GitHub Octoverse#open-source models#inference
官方账号vLLM@vllm_projectSource
ModelsUS vs ChinaMulti-sourceSelected00:37
vLLM ships day-0 support for Baidu's ERNIE 5.5 Flash Base

The benchmark gap over DeepSeek-V4-Flash-Base in the same evaluation setup makes this one of the stronger open base weights released for coding workloads, and day-0 vLLM support means teams can deploy it without custom serving work. Practitioners running RL research or domain adaptation get open base weights plus serving recipes for expert parallelism, KV offload and prefill/decode disaggregation on GB200 and H100 hardware.

#ERNIE 5.5 Flash#Baidu#vLLM#open-source models
Event
官方账号vLLM@vllm_project11 sourcesSource
PapersBuildersSelected22:33
CMU ServerBench Tests Coding Agents on LLM Serving Systems

The benchmark measures whether agents can tune the full stack, from prompts down to token serving, a capability standard coding benchmarks never touch. Teams running inference infrastructure can use it to evaluate which agent actually optimises GPU cost, and the finding that most failures came from untested code paths offers a concrete lesson for agent reliability. arXiv 2611.06841 lists all setups, so results can be reproduced independently.

#Claude Sonnet 5.5#Gemma#vLLM#ServerBench
elvis@omarsar0Source