AITOPAI热报
Industry

NVIDIA 与 SGLang 社区合作：跨五代硬件、60+ 模型零日支持

Official accountNVIDIA AI@NVIDIAAI
Post
View the post
Why we picked it

SGLang 和 vLLM 是目前自部署大模型最常用的两个推理框架，NVIDIA 这次晒出五代硬件、60 多个模型的零日适配数据，说明对 SGLang 的工程投入是真金白银。自己搭推理服务的开发者可以关注后续 Summit 的技术分享。

NVIDIA 在 SGLang Summit 上公开肯定了 SGLang（@SGL_project）与 Miles 社区对开源 AI 的推动。合作覆盖 NVIDIA Ampere 到 Vera Rubin 五代加速计算架构，累计 2 年以上联合工程工作。

Body

落地成果包括 60+ 次模型零日（day-zero）发布支持，即新模型开源当天就能在 SGLang 上推理，以及 10+ 场开发者活动和 meetup。零日支持意味着模型发布方与 SGLang 社区在模型上线前就完成适配与优化，减少开发者等待推理框架兼容的间隙。

SGLang 是主流开源 LLM 推理框架之一，与 vLLM 直接竞争服务端部署场景。NVIDIA 从硬件代际层面做深度协同，对自行部署模型的企业团队来说是降低运维与性能调优成本的信号。

Original · NVIDIA AI

NVIDIA 💚 the @SGL_project and Miles communities moving open source AI forward. From NVIDIA Ampere to Vera Rubin, our collaboration spans five generations of accelerated computing, with 2+ years of engineering together, 60+ day-zero model launches, and 10+ developer events and meetups. It's great to be at the SGLang Summit!📊 13 ⚡

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Industry" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include SGLang, NVIDIA, vLLM, open-source models.

Source attribution: curated from NVIDIA AI (@NVIDIAAI).

#SGLang#NVIDIA#vLLM#open-source models#inference frameworks

Keep reading

Recent stories with the same tag or section
ProductsBuildersOfficial06:27
Chinese contentSGLang-Neuron 上线 AWS Deep Learning Container，支持在 Trainium 上部署大模型

SGLang 用户现在可以直接把推理和 RL rollout 跑到 AWS Trainium 上，接口不变、原生 PyTorch、AWS 官方容器开箱即用。想在 GPU 之外降推理成本或统一训练推理硬件的团队可以试试这条路。

#SGLang#AWS#Trainium#inference deployment
Event
Primary sourceAWS Machine Learning Blog@Alex NailsSource
ProductsBuildersMulti-sourceSelected06:06
SGLang-Neuron brings SGLang serving to AWS Trainium

Serving stacks have mostly been GPU-bound, so a Trainium path for SGLang reduces the engineering cost of running inference on AWS's own silicon. Teams standardising on SGLang for RL rollout generation can now target Trainium instances directly. This matters for anyone evaluating Trainium against NVIDIA GPUs for cost or supply reasons, since the same APIs carry over.

#SGLang#AWS#Trainium#inference deployment
Event
Official accountAmazon Science@AmazonScienceSource
IndustryBuildersMulti-sourceSelected02:57
SGLang Consortium formed with NVIDIA, AMD, Google and Meta

Hardware vendors co-funding a single open-source inference engine reduces the risk that serving optimizations remain locked to one chip ecosystem, which matters for teams planning multi-vendor GPU fleets. Engineering leads running large-scale inference can track the consortium's roadmap to anticipate support for new models and distributed serving features. Organizations in Southeast Asia building inference capacity on heterogeneous hardware will benefit from upstream optimizations landing in SGLang rather than vendor-specific forks.

#SGLang#NVIDIA#open-source models#inference frameworks
Event
Official accountLMSYS Org (SGLang)@lmsysorgSource
ProductsBuildersSelected02:43
LMSYS releases SGLang v1.0 and Miles v0.2 at SGLang Summit

The pairing matters: a hardened v1.0 inference engine plus a production-focused RL framework gives teams a single open-source alternative to closed serving and post-training stacks. Engineers building agent or chat products on self-hosted GPUs can benchmark SGLang v1.0 against vLLM for throughput and memory gains. Teams running RL post-training can evaluate Miles v0.2 for their specific hardware mix rather than stitching together ad-hoc pipelines.

#SGLang#LMSYS#Miles#inference engines
Official accountLMSYS Org (SGLang)@lmsysorgSource
ProductsBuildersSelected00:51
AI Agents Help Integrate SGLang with IBM Spyre in a Week

The notable point is the integration velocity: a 1,500-line plugin plus AI coding agents replaced what is typically a multi-month porting effort. Teams evaluating non-GPU accelerators for LLM inference can now consider IBM Spyre as a drop-in option with vLLM-parity performance. Engineering leaders building serving infrastructure should watch how plugin-based hardware abstraction in SGLang lowers the cost of supporting new silicon.

#SGLang#vLLM#IBM Spyre#PyTorch
Official accountPyTorch@PyTorchSource
How-toBuildersSelected01:21
SGLang Megakernels Lift Muse Spark Open Decoding to 1,109 Tokens/s

Kernel-launch overhead, not compute, is often the real ceiling in multi-GPU inference, and this write-up shows concrete numbers and code proving it. Engineers building or tuning serving stacks on Blackwell-class hardware get CUDA/PTX examples and measured timelines they can adapt. Teams benchmarking against vLLM or TensorRT-LLM deployments will find the launch-count reduction a useful reference point.

#SGLang#GB300#megakernels#inference optimization
Official accountLMSYS Org (SGLang)@lmsysorgSource