DeepSeek V4.1 Flash gains 1.5x speed on GB300 NVL72
Serving-stack software, not model changes, drove most of the speedup — a signal that inference infrastructure tuning is now as decisive as raw hardware for agentic workloads. Teams benchmarking DeepSeek deployments on GB300 or comparing against NVIDIA's Dynamo + vLLM reference setup can use AA-AgentPerf numbers as a realistic baseline for agentic coding performance.
Artificial Analysis measured up to 1.5x higher tok/s per agent for DeepSeek V4.1 Flash running agentic coding workloads on NVIDIA GB300 NVL72 over the past month on its AA-AgentPerf benchmark. Single-agent speed rose from 596 to 848 tok/s, a 1.42x improvement. Per-GPU throughput also reached about 1.6x higher at roughly 80 tok/s per agent. The gains came from a newer vLLM version, new vLLM kernels for DeepSeek V4.1 on GB300, FlashInfer autotuning, fuller CUDA graph coverage, tuned speculative decoding, and co-locating prefill and decode on the same GPUs in NVIDIA's updated Dynamo + vLLM setup.
DeepSeek V4.1 Flash speeds on GB300 NVL72 have increased by up to 1.5x tok/s per agent over the past month on AA-AgentPerf, our benchmark of datacenter systems running realistic agentic coding workloads
This improvement comes from a newer vLLM version, new vLLM kernels for DeepSeek V4.1 on GB300, FlashInfer autotuning, fuller CUDA graph coverage, tuned speculative decoding, and running prefill and decode on the same GPUs in NVIDIA’s updated Dynamo + vLLM setup
➤ Faster speeds: 848 tok/s for a single agent, up from 596 (1.42x)
➤ Higher throughput: ~1.6x the throughput per GPU at ~80 tok/s per agent
