How-toMulti-sourceSelected

NVIDIA shows GLM-5.3 NVFP4 running on two DGX Stations with vLLM

Why we picked it

The practical takeaway is that frontier-size open weights can be served on just two workstation GPUs once FP4 quantisation and speculative decoding are combined, cutting the hardware bar for self-hosting. Teams evaluating local deployment versus API pricing can follow the guide directly, and it gives a named reference stack — vLLM on GB300 — to benchmark against cloud endpoints or competing setups like TensorRT-LLM.

NVIDIA published a video walkthrough and a deployment guide for running GLM-5.3 in NVFP4 quantisation locally on two DGX Stations using vLLM. The setup serves requests through a single OpenAI-compatible API and keeps all model weights resident in GPU memory.

Image · vLLM
Body

Technically, the deployment uses tensor parallelism across two GB300 GPUs. NVIDIA also enables MTP speculative decoding, an FP8 KV cache, and prefix caching to improve throughput and latency. vLLM acts as the serving layer, so teams can point existing OpenAI-compatible clients at the local endpoint without code changes.

The guide shows that GLM-5.3, a large model from Zhipu (Z.ai), can now be self-hosted on a two-node workstation footprint rather than a full data-centre cluster. For enterprises with sensitive data or unstable API access, this offers a concrete local deployment path with reference hardware and software configurations.

Original · vLLM

.@NVIDIA put together a video walkthrough and deployment guide for running GLM-5.3 NVFP4 locally on two DGX Stations with vLLM.

The setup keeps all model weights in GPU memory, using tensor parallelism across two GB300 GPUs to serve requests through a single OpenAI-compatible API. It also enables MTP speculative decoding, FP8 KV cache, and prefix caching.

Watch the walkthrough: https://t.co/Nacp2PHiC4

Get the deployment guide: https://t.co/CPnA8tK9Pw

Industry Coordinates & Source Verification

This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include GLM-5.3, NVIDIA, vLLM, DGX Station.

Source attribution: curated from vLLM (@vllm_project). Cross-source corroboration: 11 independent sources have covered this development (including Artificial Analysis, The Information, AIGCLINK), confirming industry-wide signal strength.

  • Artificial Analysis11-16 12:26Source
  • The Information11-16 22:30Source
  • AIGCLINK11-15 02:44Source
  • Aravind Srinivas11-15 03:10Source
  • arXiv cs.LG11-15 16:14Source
  • Alexandr Wang (Meta MSL)11-15 23:08Source
  • Greg Brockman11-16 00:34Source
  • 阿里通义 Qwen11-16 02:55Source
  • Hunyuan11-16 06:38Source
  • TestingCatalog11-16 09:08Source