How-to

AMD MI355X 上跑出 1,129 tok/s:sciforium 分享 DeepSeek-V4.1-Flash 推测解码实践

Why we picked it

sciforium 用客户流量微调推测解码的 drafter,让 7-token 草拟通过验证,DeepSeek-V4.1-Flash 在 4 张 AMD MI355X 上跑到 1,129 tok/s,权重还是原版没动过。做推理优化的工程师值得看,他们连失败的尝试都写出来了。

sciforium 在 4 张 AMD MI355X GPU 上将 DeepSeek-V4.1-Flash 的输出速度做到 1,129 tok/s(P90),且使用的是未经改动的原始权重,基于真实生产流量而非测试环境。

Body

核心做法在推测解码的 drafter:团队用客户流量对 drafter 做微调,让更长的草拟 token 序列能在验证环节存活,最终让 7-token 草拟长度变得划算。文章还公开了哪些优化手段没有奏效,这类负结果在工程分享中较少见。

此前外界曾质疑 AMD GPU 的推理性能是否跟得上,这次纪录是在 AMD 硬件上以生产流量跑出来的,对关心非 NVIDIA 推理栈的工程团队有直接参考价值。

Original · elvis

Great read on speculative decoding in practice. The interesting part is the drafter. @sciforium tunes it on customer traffic, so longer runs of drafted tokens survive verification, which is what made 7-token drafts worth it. Result: DeepSeek-V4.1-Flash at 1,129 output tok/s on 4 AMD MI355X GPUs, with the original weights. They also share what didn't help, which I appreciate. Hassan Akbari @hassanience not long ago, people questioned whether AMD GPUs could keep up. today, we’re setting records on them. 1,129 output tps at P90. Real production traffic, not a toy project. a little faith, a lot of engineering. here’s how we did it. x.com/i/article/2122… 🔗 View Quoted Tweet📊 7 ⚡

Industry Coordinates & Source Verification

This report is categorized under AITOP's "How-to" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include DeepSeek-V4.1-Flash, AMD MI355X, speculative decoding, inference optimization.

Source attribution: curated from elvis (@omarsar0).