这篇论文发现Muon优化器让智能体RL成功率从29%飙到55%，比AdamW快60步，还分析了学习率的影响。
Muon优化器在稀疏奖励智能体强化学习中展现优势。使用Qwen2.5-0.5B-Instruct和ALFWorld环境，在Group-in-Group Policy Optimization (GiGPO)下，仅将Muon应用于隐藏权重矩阵，最终窗口验证成功率从0.290提升至0.546（+88%），而高学习率AdamW无后更新成功。在3e-5学习率下，Muon将GRPO成功率从0.161提升至0.268；在1e-5下，GraphGPO Muon达到0.901，归一化验证AUC从0.399增至0.556，并提前30和60轮达到0.5和0.75成功率。该结果表明Muon可提升智能体强化学习性能，但效果依赖优势估计器和学习率。
When Does Muon Help Agentic Reinforcement Learning?
Muon is competitive with AdamW in large-scale pre-training, but its value for reinforcement-learning (RL) post-training remains unclear. We study vanilla Muon in sparse-reward agentic RL through matched single-seed comparisons with AdamW on ALFWorld using Qwen2.5-0.5B-Instruct. Under Group-in-Group Policy Optimization (GiGPO), applying Muon only to hidden weight matrices raises final-window validation success from 0.290 to 0.546 (+88%); high-rate AdamW controls retain no post-update success. The effect depends on the advantage estimator and learning rate. At 3e-5, Muon improves GRPO from 0.161 to 0.268, whereas GraphGPO's late-window gap narrows near saturation. At 1e-5, GraphGPO Muon reaches 0.901, raises normalized validation AUC from 0.399 to 0.556, and reaches 0.5 and 0.75 success 30 and 60 updates earlier, respectively. These exploratory results show that Muon can benefit agentic RL and motivate studying the policy optimizer, advantage estimator, and learning rate jointly. Multi-seed and cross-task validation remain open.