这篇论文揭示了有效学习率在语言模型预训练中的关键作用，通过分析ELR与损失动态的关系，为优化预训练过程提供了新的视角。与传统的学习率相比，ELR提供了更精确的控制，有助于提高模型性能。
发现语言模型预训练中的ELR崩溃：学习率（LR）和参数范数主要通过它们的比率，即有效学习率（ELR）来主导损失动态。当ELR在运行间匹配时，尽管LR和参数范数存在显著差异，但它们的损失轨迹在整个训练过程中都会崩溃。在优化器、架构、数据集和模型规模方面，平均崩溃误差通常为几个x 10^-3，低于在代表性配置中测量的种子到种子的变化。系统性的消融识别了归一化设计和LR-范数变化的时标作为崩溃精度的关键决定因素。受控干预进一步表明，权重衰减和Hyperball主要通过它们诱导的ELR计划来塑造损失动态。用ELR替换LR使拟合的功能缩放定律（FSL）能够在范数控制方法之间迁移。基于ELR的FSL还解释了延迟加速，这是范数控制的一个常见效果。这些结果共同确立了ELR作为连接LR调度、范数控制和损失动态的共同坐标。
We uncover ELR collapse in language model pretraining: learning rate (LR) and parameter norm govern loss dynamics primarily through their ratio, the effective learning rate (ELR). When ELR is matched across runs, their loss trajectories collapse throughout training despite substantially different LRs and parameter norms. Across optimizers, architectures, datasets, and model scales, mean collapse errors are typically a few x 10^-3, below the seed-to-seed variation measured in a representative configuration. Systematic ablations identify normalization design and the timescale of LR-norm variation as key determinants of collapse precision. Controlled interventions further show that weight decay and Hyperball shape loss dynamics primarily through the ELR schedules they induce. Replacing LR with ELR enables a fitted functional scaling law (FSL) to transfer across norm-control methods. The resulting ELR-based FSL also explains delayed acceleration, a recurring effect of norm control. Together, these results establish ELR as a common coordinate linking LR scheduling, norm control, and loss dynamics.