SERA:按谱能量自适应分配秩的模型合并方法
Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging
arXiv 新论文提出 SERA,合并多个微调模型时按任务分配秩预算,总预算不变还能涨点。
论文提出 SERA(Spectral Energy-proportional Rank Allocation),一种按奇异值能量结构自适应分配秩的模型合并策略。现有基于 SVD 的谱合并方法给每个任务向量分配相同的秩容量,忽视了任务向量之间谱复杂度的差异。SERA 给谱结构复杂或孤立的任务分配更多方向,给紧凑任务分配更少方向,把 SVD 合并从统一容量扩展到任务相关容量分配。在标准视觉模型合并协议下,SERA 在保持与现有谱方法相同总秩预算的同时提升了多任务合并性能。分析还表明任务级谱集中度与自适应秩分配的单任务收益相关,可据此判断 SERA 在哪些任务上更有效。
Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging
Model merging aims to combine multiple fine-tuned models derived from a common pretrained model into a single multi-task model without additional joint training. Recent spectral merging methods improve over simple weight averaging by exploiting low-rank structures of task-specific updates, but they commonly assign the same rank capacity to every task. This uniform allocation ignores that task vectors can have heterogeneous spectral complexity, causing the shared merging space to be used suboptimally. In this paper, we propose Spectral Energy-proportional Rank Allocation (SERA), a simple task-adaptive strategy that allocates ranks according to the singular-value energy structure of each task vector. By assigning richer spectral capacity to complex or isolated tasks and fewer directions to compact tasks, SERA extends SVD-based model merging from uniform-capacity merging to task-dependent capacity allocation. Experiments under standard vision model merging protocols show that SERA improves multi-task merging performance while preserving the same total rank budget as existing spectral merging methods. Further analysis demonstrates that task-level spectral concentration is closely related to the per-task effect of adaptive rank allocation, providing insight into when and why SERA is effective.