#PD分离

1 stories · 1 in 7 days · 1 in 30 days
Stories, products and papers tagged "PD分离", newest first.
Nov 19
How-toBuildersSelected18:58
Ant International serves DeepSeek-V4.1-Flash on 8x H200 via SGLang

The non-obvious takeaway is the explicit cost curve: prefill speed and TTFT gains of up to 64% come at a measurable price of 3%-52% higher TPOT and a 1.36-point GSM8K drop, so the 128-token late-layer mode fits latency-sensitive chat, not accuracy-critical decoding. Engineers running DeepSeek or MoE models on H200 clusters can copy the SGLang PD disaggregation recipe directly. Given Ant International is Singapore-headquartered, the write-up is a concrete datapoint on how regional infrastructure teams optimize open-weight Chinese models on US GPUs.

Event
Official accountLMSYS Org (SGLang)@lmsysorgSource