Pion 解决了传统优化器在训练中破坏权重矩阵谱结构的问题，做 LLM 训练的研究者和工程师值得关注，尤其适合追求训练稳定性和模型质量的团队。
Pion 是一种用于大语言模型训练的新型优化器，通过左右正交变换更新权重矩阵，在整个训练过程中保持其奇异值不变。与 Adam 和 Muon 等加法优化器不同，Pion 在固定权重矩阵谱范数的同时调节其几何结构，提供了一种稳定的优化机制。实验表明，Pion 在 LLM 预训练和微调中均能作为标准优化器的稳定且有竞争力的替代方案。
Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
We introduce Pion, a spectrum-preserving optimizer for large language model (LLM) training based on orthogonal equivalence transformation. Unlike additive optimizers such as Adam and Muon, Pion updates each weight matrix through left and right orthogonal transformations, preserving its singular values throughout training. This yields an optimization mechanism that modulates the geometry of weight matrices while keeping their spectral norm fixed. We derive the Pion update rule, systematically examine its design choices, and analyze its convergence behavior along with several key properties. Empirical results show that Pion offers a stable and competitive alternative to standard optimizers for both LLM pretraining and finetuning.