这篇论文提出用自动任务排序加独立QLoRA来微调LLM，在TRACE基准上拿到44.78，比堆参数容量更管用。
参数高效微调（PEFT）通常使用单一共享LoRA适配器，在异构任务序列上会产生干扰和灾难性遗忘。现有方法通过增加参数容量或组合多个适配器来提升表现，但仍依赖共享优化路径。本文提出自动多策略PEFT框架，通过任务分组和排序组织优化路径，采用独立QLoRA实现异构任务解耦优化。在TRACE基准上，该框架在相同可训练容量下达到44.78的最佳性能，验证了优化路径组织比单纯增加适配器容量更有效。
The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs
Parameter-Efficient Fine-Tuning (PEFT) commonly adapts large language models using a single shared Low-Rank Adapter (LoRA). This shared optimization space often suffers from interference when adapting heterogeneous task sequences, leading to poor transfer and catastrophic forgetting. Existing approaches mainly improve adapter expressiveness by increasing parameter capacity or composing multiple adapters, yet they still rely on a shared optimization path. In this paper, we propose an optimization-path organization framework for parameter-efficient fine-tuning of large language models, implemented as an automatic multi-policy PEFT architecture. Specifically, optimization-compatible adaptation paths are automatically organized through task grouping and task sequencing under a fixed parameter budget. The organized optimization paths are implemented as independent Quantized Low-Rank Adapters (QLoRA), enabling heterogeneous tasks to be optimized in decoupled adaptation spaces while preserving positive transfer among compatible tasks. Experiments on the TRACE benchmark demonstrate that performance consistently improves from conventional single-policy PEFT to multi-policy PEFT, with the proposed automatic multi-policy framework achieving the best performance of 44.78 under the same trainable capacity. This suggests that optimization-path organization is more effective than simply increasing adapter capacity for heterogeneous parameter-efficient fine-tuning.