NoRA改进LoRA训练，不增加参数却能加速收敛、提升稳定性，适用于多种训练场景。
研究团队提出Normalized Low-Rank Adaptation (NoRA)方法，通过标准化下投影矩阵改进LoRA训练。NoRA在预训练、监督微调和强化学习中均表现优异，加速收敛速度，提升性能和训练稳定性，并缓解灾难性遗忘。该方法无需额外可训练参数或推理时计算，是对LoRA的简单有效增强。
Normalized Low-Rank Adaptation
While low-rank adaptation (LoRA) is widely used for parameter-efficient model adaptation, how to regularize its training dynamics for stable and effective optimization remains underexplored. Because LoRA initializes the up-projection to zero, its early optimization dynamics are largely governed by the down-projection. Building on this observation, we introduce Normalized Low-Rank Adaptation (NoRA), a simple yet effective method that normalizes the down-projection matrices during training. We further show that the same normalization can be applied only at initialization, improving standard LoRA without requiring repeated normalization throughout training. Across pretraining, supervised finetuning, and reinforcement learning, NoRA consistently accelerates convergence, improves performance and training stability, and mitigates catastrophic forgetting. These benefits require neither additional trainable parameters nor inference-time computation, making NoRA a simple and broadly applicable enhancement to LoRA.