AITOPAI热报
ProductsMulti-sourceSelected71°

vLLM v0.33.0 Ships PyTorch 2.14, New Models and NVIDIA Gains

官方账号vLLM@vllm_project
Post· 11 sources
View the post
Why we picked it

The ABI-stable libtorch build and CUDA 13.0-only wheels mean teams pinned to older CUDA versions must plan their upgrade path before pulling this release. Serving operators running NVIDIA GB300 or AMD GPUs gain measurable latency and throughput wins, and inference engineers can adopt the new OCR and reasoning models immediately. Step-5 Preview support also signals tighter integration with Chinese frontier models for self-hosted deployments.

vLLM v0.33.0 was released with 927 commits from 389 contributors, including 122 first-timers. The release adds support for four new models: Reflection Beam-1.0, Step-5 Preview, Nemotron Parse 3.5 and MiniCPM-OCR 1.0. It moves to PyTorch 2.14 with CUDA 13.0-only wheels and a libtorch ABI-stable build. NVIDIA performance improved with 1.5-2x decode latency gains on GB300 and 39-48% lower TTFT, while ROCm 10.0 becomes the default for AMD. Speculative decoding additions include XPress refiner and FLy lossy verify with 1.04-1.31x speedups.

Original · vLLM

vLLM v0.33.0 is out! 🚀

927 commits · 389 contributors · 122 first-timers

🤖 New models: Reflection Beam-1.0, Step-5 Preview, Nemotron Parse 3.5, MiniCPM-OCR 1.0 🔥 PyTorch 2.14 — CUDA 13.0-only wheels, libtorch ABI-stable 🎲 Spec decoding: XPress refiner, FLy lossy verify (1.04–1.31x), DSpark 1373→3390 tok/s on GB300 ⚡ NVIDIA decode: 1.5–2x latency decode on GB300, FA4 sparse MLA, 39–48% lower TTFT 🚀 New AWS EFA image + shorter vllm/vllm name 🐧 ROCm 10.0 default, MonoKernel 2.32x P50

Full notes ➡️ https://t.co/nB301yri6u

#vLLM#PyTorch#inference#open-source models#speculative decoding

Keep reading

Recent stories with the same tag or section
How-toBuildersSelected10:19
LlamaIndex speeds up OCRia-1 token generation 10× on a B200

The instructive part is the method: no single trick delivered the win, so the blog's testing discipline for stacking small, accuracy-checked optimizations is transferable to almost any inference workload. Builders serving document models or running vLLM can mine the writeup for concrete techniques they can replicate on their own GPUs.

#LlamaIndex#OCRia-1#vLLM#inference optimization
Event
Jerry Liu@jerryjliu02 sourcesSource
ModelsBuildersSelected04:01
LlamaIndex speeds up OCRia-1 doc parsing 10x on one B200

The practical lesson is that compounding many small, accuracy-checked optimizations can deliver a 10x throughput gain without new hardware, and the details are documented in a public blog post. Inference engineers and teams serving document pipelines can study the writeup to cut their own GPU costs. Since some improvements will land in open-source vLLM, the gains will eventually benefit anyone serving models, not just LlamaParse users.

#OCRia-1#LlamaIndex#vLLM#inference optimization
Event
Jerry Liu@jerryjliu02 sourcesSource
ProductsBuildersMulti-sourceSelected01:08
LlamaIndex speeds up OCRia-1 inference nearly 10× on one B200

The compounding effect of 17 small inference tweaks, rather than a single big fix, is the lesson here for anyone serving document models in production. The team's methodology of testing every change against both throughput and accuracy is reusable for other inference stacks. Contributions landing in vLLM will eventually benefit anyone running that framework directly.

#OCRia-1#LlamaIndex#vLLM#NVIDIA B200
Event
LlamaIndex@llama_index11 sourcesSource