Muon 优化器在鲁棒性和迁移性上全面超越 Adam，做预训练或迁移学习的团队值得关注，尤其是需要模型对噪声和下游任务更鲁棒的场景。
Muon 作为预训练大语言模型和视觉分类器的新兴优化器，其效率优势已超过 Adam 和 SGD，但特征学习优势尚不明确。本文通过鲁棒性和迁移性视角研究 Muon 的特征学习优势：在损坏图像和文本上评估预训练模型，发现 Muon 学到的特征比 Adam 和 SGD 更鲁棒，且这种优势体现在更大的 logit 边际上。通过下游任务的线性分类器或微调，Muon 的特征迁移效果更好，这得益于隐藏状态的有效秩更高。在含多组件的分类问题中，论文从理论上证明了 Muon 能获得更大边际和更高有效秩。
Muon Learns More Robust and Transferable Features than Adam
Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs) and vision classifiers. Despite its efficiency advantage over Adam and SGD, the feature-learning advantage of Muon remains unclear. This paper investigates Muon's feature-learning advantage through the lens of robustness and transferability. First, by evaluating pretrained models on corrupted images and texts, we show that features learned by Muon are consistently more robust than those learned by Adam and SGD across different architectures, including transformers and Convolutional Neural Networks (CNNs). Using trained layer-wise probes, we further show that this robustness advantage is reflected in larger logit margins across layers. Second, by training linear classifiers or fine-tuning full models from pretrained parameters on downstream tasks, we demonstrate that Muon-learned features transfer more effectively than those learned by Adam and SGD. This transferability advantage is further supported by the diversity of hidden states across layers, as measured by effective rank. Finally, in a representative classification problem with multi-component features, we prove that Muon attains larger margins and higher effective rank than Adam and SGD, providing theoretical support for our empirical findings.