DeepSeek-V4 Flash RL 在 AMD 显卡上跑通了，训练结果还不错，AIME 分数涨了一截。想试试 AMD 平台搞 RL 训练可以看看这篇。
LMSYS 发布博客，介绍 DeepSeek-V4 Flash RL 在 AMD Instinct MI355X GPU 上使用 Miles 框架完成端到端训练。训练100+步后，log-prob gap 稳定在0.09，AIME-2024 pass@1 从0.39提升至0.49，pass@8 从0.53提升至0.67。采用 FP8 推演加 BF16 演员模型，支持数据类型感知在线权重更新。在 ROCm 上实现了稳定的 TP1/PP4/EP4 布局，无集体通信停顿。混合注意力、mHC 混合和哈希路由 MoE 在 SGLang 和 Megatron 引擎间保持一致。
🚀 New blog: Bringing DeepSeek-V4 Flash RL Training…
🚀 New blog: Bringing DeepSeek-V4 Flash RL Training to AMD Instinct MI355X GPUs with Miles
DeepSeek-V4 Flash RL now runs end-to-end in Miles on @AMD Instinct MI355X GPUs with ROCm! Together with AMD, we aligned the model behavior across SGLang rollout and Megatron training, validated over 100+ optimizer steps on four MI355X nodes:
1️⃣ Train-rollout log-prob gap bounded at ~0.09 across 100+ steps 2️⃣ AIME-2024 pass@1 rose from 0.39 to 0.49, pass@8 from 0.53 to 0.67 3️⃣ FP8 rollout + BF16 actor with datatype-aware online weight updates 4️⃣ Stable TP1 / PP4 / EP4 layout on ROCm without collective stalls 5️⃣ Hybrid attention, mHC mixing & hash-routed MoE aligned across both engines