How-toSelected

实测对比:RoCE 与 Quantum InfiniBand 推理性能差距明显

Why we picked it

Lambda 在 2 台 HGX B200 上跑了组对照实验,其他条件全一样,只换网络:RoCE 比 InfiniBand 吞吐低 70%,首 token 慢 2.5 倍。正在搭推理集群的工程师选网络方案前应该看看这组数据,省得踩坑。

Lambda 用 2 个 HGX B200 节点部署同一模型 nvidia/GLM-5.2-NVFP4,在约 250 并发用户下测试:RoCE 方案的吞吐量比 NVIDIA Quantum InfiniBand 低约 70%,首 token 延迟慢 2.5 倍。

Image · Lambda
Body

测试控制了其余变量:两边都跑 SGLang TP16/DP16/EP16 配置,基准统一用 AIPerf(8192 输入 / 1024 输出 token),唯一的变量是互联网络。但作者也指出互联并非唯一影响因素,未完全展开其他变量。

这组数字对正在选型大规模推理集群的团队有直接参考意义:以太网 RoCE 虽然成本更低,但在高并发下与 InfiniBand 的性能差距可能远超预期。企业在规划 B200 这类多节点推理部署时,需要把互联方案的 throughput 和 TTFT 差距计入总成本评估。

Original · Lambda

At roughly 250 concurrent users on 2x HGX B200 nodes, a RoCE deployment delivered about 70% lower throughput and 2.5x slower time to first token than NVIDIA Quantum InfiniBand.

Same model (nvidia/GLM-5.2-NVFP4), same serving framework (SGLang TP16/DP16/EP16), same benchmark (AIPerf, 8192 in / 1024 out tokens). The interconnect was the variable, but not the only one.

https://t.co/0YDhLXFLCO

Industry Coordinates & Source Verification

This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include InfiniBand, RoCE, SGLang, HGX B200.

Source attribution: curated from Lambda (@LambdaAPI).