AITOPAI热报
ProductsMulti-source

vLLM 11月12日赴马德里办开源推理之夜

官方账号vLLM@vllm_project
Post· 11 sources
View the post
Why we picked it

vLLM 11月12日和 Red Hat、NVIDIA 在马德里办活动，讲 vLLM、vLLM-omni、LLM Compressor 和 K8s 上的 ll-d，做推理部署的可以去看看。

vLLM 项目宣布 11 月 12 日在马德里与 Red Hat AI、NVIDIA 联合举办开源推理主题活动。内容覆盖 vLLM、vLLM-omni、LLM Compressor 和在 Kubernetes 上运行的 llm-d。活动提供公开报名链接，面向对推理部署感兴趣的开发者。

Image · vLLM
Original · vLLM

Hola Madrid! 🇪🇸 vLLM is coming to town — join us Nov 12 with @RedHat_AI and @NVIDIAAI for an evening of open-source inference: vLLM, vLLM-omni, LLM Compressor, and llm-d on Kubernetes. Register: https://t.co/zstjWbYNcp

#vLLM#llm-d#Red Hat#NVIDIA#推理优化

Keep reading

Recent stories with the same tag or section
How-toBuildersSelected10:19
LlamaIndex speeds up OCRia-1 token generation 10× on a B200

The instructive part is the method: no single trick delivered the win, so the blog's testing discipline for stacking small, accuracy-checked optimizations is transferable to almost any inference workload. Builders serving document models or running vLLM can mine the writeup for concrete techniques they can replicate on their own GPUs.

#LlamaIndex#OCRia-1#vLLM#inference optimization
Event
Jerry Liu@jerryjliu02 sourcesSource
ProductsBuildersMulti-sourceSelected71°10:00
vLLM v0.33.0 Ships PyTorch 2.14, New Models and NVIDIA Gains

The ABI-stable libtorch build and CUDA 13.0-only wheels mean teams pinned to older CUDA versions must plan their upgrade path before pulling this release. Serving operators running NVIDIA GB300 or AMD GPUs gain measurable latency and throughput wins, and inference engineers can adopt the new OCR and reasoning models immediately. Step-5 Preview support also signals tighter integration with Chinese frontier models for self-hosted deployments.

#vLLM#PyTorch#inference#open-source models
Event
官方账号vLLM@vllm_project11 sourcesSource
ModelsBuildersSelected04:01
LlamaIndex speeds up OCRia-1 doc parsing 10x on one B200

The practical lesson is that compounding many small, accuracy-checked optimizations can deliver a 10x throughput gain without new hardware, and the details are documented in a public blog post. Inference engineers and teams serving document pipelines can study the writeup to cut their own GPU costs. Since some improvements will land in open-source vLLM, the gains will eventually benefit anyone serving models, not just LlamaParse users.

#OCRia-1#LlamaIndex#vLLM#inference optimization
Event
Jerry Liu@jerryjliu02 sourcesSource
ProductsBuildersMulti-sourceSelected01:08
LlamaIndex speeds up OCRia-1 inference nearly 10× on one B200

The compounding effect of 17 small inference tweaks, rather than a single big fix, is the lesson here for anyone serving document models in production. The team's methodology of testing every change against both throughput and accuracy is reusable for other inference stacks. Contributions landing in vLLM will eventually benefit anyone running that framework directly.

#OCRia-1#LlamaIndex#vLLM#NVIDIA B200
Event
LlamaIndex@llama_index11 sourcesSource