分布式训练团队终于有了异步 LMO 方法的理论保障——Ringmaster LMO 解决了异构集群中慢节点拖累效率的问题，做大规模预训练或异构系统优化的开发者值得关注。
Muon 作为 AdamW 的替代方案在神经网络训练中表现出色，但基于线性最小化预言机（LMO）的方法通常采用同步训练，在异构分布式系统中受限于慢速工作节点。本文提出 Ringmaster LMO，一种异步 LMO 动量方法，借鉴 Ringmaster ASGD 的延迟阈值思想，通过丢弃过时梯度实现最优时间复杂度。该方法在广义 (L0, L1)-光滑性下建立了收敛保证，并开发了参数无关的变体。实验表明，在随机二次问题和 NanoChat 语言模型预训练中，Ringmaster LMO 在异构环境下显著优于同步和异步基线。
Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
Muon has recently emerged as a strong alternative to AdamW for training neural networks, with encouraging large-scale pretraining results and growing evidence that matrix-structured updates can be faster in practice. Yet Muon, and more generally Linear Minimization Oracle (LMO) based methods, are typically used synchronously. This is problematic in heterogeneous distributed systems, where workers complete gradient computations at different speeds and synchronous training must repeatedly wait for slower workers. In this work, we introduce Ringmaster LMO, an asynchronous LMO-based momentum method for unconstrained stochastic nonconvex optimization. Our method builds on the delay-thresholding idea of Ringmaster ASGD. For SGD-type methods, Ringmaster ASGD achieves optimal time complexity by discarding overly stale gradients. Ringmaster LMO extends this mechanism to general LMO-based updates. We establish convergence guarantees under generalized $(L_0, L_1)$-smoothness and further develop a parameter-agnostic variant with decreasing stepsizes and adaptive delay thresholds. Finally, we translate our iteration guarantees into time complexity bounds under heterogeneous worker computation times. In the classical Euclidean smooth setting, these bounds recover the optimal time complexity of Ringmaster ASGD. Experiments on stochastic quadratic problems and NanoChat language-model pretraining show that the advantages of Ringmaster LMO grow with system heterogeneity and that the method outperforms strong synchronous and asynchronous baselines.