ModelsOfficial
摩尔线程开源 Megatron-MUSA-Titan，支持万亿参数 MoE 训练
Moore Threads Open-Sources Megatron-MUSA-Titan for Trillion-Parameter MoE Training on Its GPUs
Why we picked it
摩尔线程把自家万卡集群训万亿参数 MoE 的训练栈开源了，带块级 FP8，国产 GPU 上也能跑 Megatron 式大规模训练。
摩尔线程开源了 Megatron-MUSA-Titan 训练框架，这是其用于在 10000 卡 GPU 集群上进行万亿参数 MoE 预训练的内部训练栈。该框架采用块级 FP8（blockwise FP8）技术，并支持 slime 和 miles 两个 RL 训练工具。这意味着开发者可以在摩尔线程国产 GPU 上复用 NVIDIA Megatron 生态的大规模训练方案。
Original · pandaily
Moore Threads Open-Sources Megatron-MUSA-Titan for Trillion-Parameter MoE Training on Its GPUs
Moore Threads has open-sourced Megatron-MUSA-Titan, the training stack it uses for trillion-parameter MoE pretraining on 10,000-GPU clusters, with blockwise FP8 and support for slime and miles RL.
