BISECT：双层谱分解让 LoRA 持续学习突破零空间隔离瓶颈
BISECT: Bilevel Spectral Decomposition for Low-Rank Continual Learning
一帮研究者发现 LoRA 持续学习的零空间隔离会饿死新任务学习，提出 BISECT 用谱分解解决，在五个基准上跑赢旧方法还零推理开销。
论文指出主流基于零空间隔离的 LoRA 持续学习方法存在"谱饥饿"瓶颈：下投影基在任务优化前锁定后，历史零空间尾部仅能覆盖新任务方差的约 5.5%，导致新类可塑性不足。BISECT 将固定 rank-r 预算划分为两个正交功能块，分别承载共享主模式和通过广义瑞利白化提取的新颖性补集。训练后用闭式精度门把两个块折回冻结主干，按瑞利比率缩放每个 rank-1 不变原子。在 ImageNet-R、ImageNet-A、CIFAR-100、CUB-200、DomainNet 的八个增量设置上，BISECT 超越 prompt、adapter 和 LoRA 基线，推理阶段零额外参数和 FLOPs。
BISECT: Bilevel Spectral Decomposition for Low-Rank Continual Learning
Exemplar-free continual learning on pre-trained Vision Transformers increasingly relies on Low-Rank Adaptation (LoRA), yet prevailing methods govern the low-rank update space through strict subspace isolation. By projecting each incoming task into the approximate null space of historical token activations, isolation-based learners equate any cross-task feature overlap with catastrophic interference. We show that this orthogonality premise induces a structural bottleneck we term \emph{spectral starvation}: when down-projection bases are locked before task optimization, the gradient norm reaching the trainable up-projection is upper-bounded by the energy of current-task features captured by the chosen basis. Because natural visual tasks share low- and mid-level covariance structure, historical null tails contain barely $5.5\%$ of incoming task variance, starving new-class plasticity while blocking positive consolidation along shared directions. We propose BISECT (Bilevel Spectral Decomposition), a continual LoRA framework that partitions a fixed rank-$r$ budget into two orthogonal functional blocks: an invariant carrier spanned by the dominant modes of the joint historical-and-current Gram matrix, and a novelty complement extracted via generalized Rayleigh whitening followed by orthogonal complement projection. After single-task training, BISECT folds both blocks back into the frozen backbone using a closed-form precision gate that scales each rank-1 invariant atom by its current-to-joint Rayleigh ratio while preserving novelty atoms at unit gain. Across eight incremental settings on ImageNet-R, ImageNet-A, CIFAR-100, CUB-200, and DomainNet, BISECT outperforms prompt, adapter, and LoRA baselines with zero extra inference parameters or FLOPs, and remains complementary to sparse analytic expansion.