How-toSelected

Ant Intl 在 8×H200 上用 SGLang 优化 DeepSeek-V4.1-Flash 推理

Why we picked it

蚂蚁国际把 DeepSeek-V4.1-Flash 在 8 张 H200 上的推理优化思路完整公开了:稀疏注意力、大 Prefill 分块、后层只算 128 个 token,TTFT 最高降 79%。代价也写清楚了——TPOT 变慢、GSM8K 掉约 1.4 个点。自己搭推理服务的人值得照着调一遍。

Ant International 的 AI Infra 团队发布博客,介绍如何在 8×H200 上以 SGLang PD 分离架构部署 DeepSeek-V4.1-Flash。在 8 组测试场景(64K 输入/512 输出、前缀命中率 50/70/90%、并发 4–32)下,端到端延迟较原基线降低 15%–64%,TTFT 降低 41%–79%。

Body

技术手段包括:采用成本更低的稀疏注意力 Indexer、加大 Prefill 分块、精简 MoE 通信,以及让靠后的层只计算最后 128 个 token——该 128-token 模式可再压低 TTFT 最高 46.7%。

代价是 TPOT 升高 3%–52%,GSM8K 成绩从 96.74% 降至 95.38%。博客展示了在 H200(非 H20/国产卡)上跑满 DeepSeek 推理性能的具体路径,对自建推理服务的团队有直接参考价值。

Original · LMSYS Org (SGLang)

Optimizing DeepSeek-V4.1 on H200 with SGLang PD Disaggregation

The AI Infra team at @Ant_Intl served DeepSeek-V4.1-Flash from @deepseek_ai on 8×H200 with PD disaggregation.

This blog walks through how they tackled it: a cheaper sparse-attention Indexer, bigger Prefill chunks, leaner MoE communication, and late layers that compute only the last 128 tokens.

Highlights: · E2E 15%–64% lower, TTFT 41%–79% lower vs the original baseline, across 8 cases (64K in / 512 out, 50/70/90% prefix hit, concurrency 4–32) · The 128-token mode cuts TTFT by up to a further 46.7% · The cost: TPOT 3%–52% higher, GSM8K 96.74% → 95.38%

Read the full blog below

Industry Coordinates & Source Verification

This report is categorized under AITOP's "How-to" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include DeepSeek, SGLang, H200, inference optimization.

Source attribution: curated from LMSYS Org (SGLang) (@lmsysorg).