UTA 学习率衰减方法发布，适配 Adam 型优化器
UTA: Universal Training Adaptive Learning Rate Decay
训练模型调学习率衰减的可以看看，UTA 比固定衰减曲线更聪明，能跟着训练状态走，LLaMA 和 ResNet 上都验证了，直接即插即用。
论文提出 Universal Training Adaptive（UTA）学习率衰减方法，改进此前的 Universal Budget Aware（UBA）框架。UTA 将学习率衰减建模为闭环的形状与尺度优化问题，通过连续时间公式使衰减路径随训练状态自适应调整。实验覆盖 LLaMA 60M 到 1B 的语言模型，以及在 ImageNet 上训练的 ResNet50 和 ViT-B 视觉模型，均取得一致提升。该方法即插即用，额外开销可忽略，消融实验验证了其在峰值学习率、衰减比例和微调场景下的稳健性。
UTA: Universal Training Adaptive Learning Rate Decay
Learning rate decay (LRD) is critical for training high-performance models, yet most designs remain empirically driven. Although the Universal Budget Aware (UBA) LRD offers a theoretical framework, its reliance on SGD dynamics limits its applicability to modern Adam-type optimizers and leaves its key `optimization difficulty parameter' opaque. To bridge this gap, we propose the Universal Training Adaptive (UTA) LRD. Unlike UBA's fixed offline trajectory anchored solely to iterations, UTA treats LRD as a closed-loop shape-and-scale problem. By continuous-time formulation, UTA is training-state adaptive with segment-wise optimal shape. This adaptivity links external iterations with internal training states, enabling a near-optimal path in both progression shape and learning rate (LR) scale, a property unattainable by other LRDs, including UBA. Empirically, we demonstrate consistent improvements across language models (LLaMA 60M to 1B) and vision models (ResNet50, ViT-B) on ImageNet with plug-and-play ease and negligible overhead. Extensive ablation studies further confirm UTA's robustness across dimensions, including peak LR, final decay ratio, its parameter $\varphi_0$, component efficacy and generalization to fine-tuning scenarios, underscoring its practical implementability.