DeepSpeed adds AutoSP, Muon support and auto parallelism at PyTorchCon
AutoTP and AutoEP applied to Hugging Face models mean teams can distribute training without hand-tuning tensor or expert parallelism, lowering the barrier for mid-size labs. Practitioners running multi-vendor accelerator fleets gain a portable optimization stack outside vendor-locked toolchains. As DeepSpeed is open source, the features are immediately available to ASEAN startups building training infrastructure on limited budgets.
Zhipeng Wang of Google presented DeepSpeed team advances at PyTorchCon 2026 on 2-scale large model training. The update introduces AutoSP for automatic sequence parallelism, Muon optimizer support, and AutoTP and AutoEP for Hugging Face models. Wang framed these as model-systems co-design optimizations that keep performance high across diverse model architectures, workloads and accelerator platforms. He also covered the DeepSpeed open source community's ongoing collaboration with model developers, hardware vendors and AI infrastructure partners.
Zhipeng Wang (@Google) shares technical advances from the @DeepSpeedAI team for large-scale model training at #PyTorchCon 2026.
Zhipeng highlights model-systems co-design and optimizations that enable DeepSpeed to deliver high performance across diverse model architectures, workloads, and accelerator platforms. Highlights include AutoSP for automatic sequence parallelism, Muon optimizer support, and AutoTP and AutoEP for Hugging Face models.
He discusses the continued development of the DeepSpeed open source community and collaboration with model developers, hardware vendors, and AI infrastructure partners to expand adoption and enable emerging large-scale AI workloads.
