这篇论文用数学公式解释了LoRA微调为什么会破坏模型原有知识，而且实验覆盖了9000多层，结果很扎实。如果你做LoRA微调，能帮你预测什么时候会出现遗忘。
该论文提出了一种谱分布定律，用于预测LoRA微调中出现的入侵维度（intruder dimensions），即与预训练奇异向量几乎正交的新奇异向量，这些向量会导致灾难性遗忘。通过推导每层临界更新强度s*=θ̄/(γσ₁(BA))，基于矩形尖峰变形变换（rectangular spiked-deformation transform）和精确的谱方程，无需拟合参数即可预测。在涵盖18个适配器、9840层扫描的实验中，该定律在82%的层上将阈值定位在2倍误差内，区分入侵层与非入侵层的平均AUC为0.89。结合两种预定的边缘评估，准确率达98%，并在第三方适配器上验证为0.997。基于阈值推导的尖峰预算规则（spike-budget rule）仅需一次SVD，无需验证扫描，即可在最脆弱的模型上将遗忘减少62%。
The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning
LoRA fine-tuning can create intruder dimensions: new leading singular vectors of the updated weight matrix $W+BA$ that are nearly orthogonal to all pretrained singular vectors and that drive catastrophic forgetting. Since their discovery, no theory has predicted, layer by layer on measured spectra, when they appear. We derive a per-layer critical update strength $s^\ast=\barθ/(γσ_1(BA))$, computed from the measured spectrum of $W$ alone through the rectangular spiked-deformation transform, together with an exact secular-equation characterization of the updated spectrum, with no fitted parameters. In a pre-specified study spanning four dense Transformer families, a state-space model, a mixture-of-experts model, and an encoder-decoder (18 adapters, 9{,}840 layer scans), the law localizes the empirical threshold within a factor of two on $82\%$ of layers, separates intruder-bearing from intruder-free layers at deployment with a mean AUC of $0.89$, holds unchanged on six third-party adapters, and predicts where WikiText-2 perplexity begins to degrade; a combination of the two pre-specified edge evaluations reaches $98\%$ and is confirmed out-of-bag on the external adapters ($0.997$). Full fine-tuning disperses its update far below the threshold of every layer, which resolves the asymmetry between LoRA and full fine-tuning. Norm-matched interventions confirm that threshold-crossing layers, rather than update magnitude, carry the forgetting, and a spike-budget rule derived from the thresholds, requiring one SVD and no validation sweeps, reduces forgetting by $62\%$ on the most fragile model at no task cost.