这篇论文重新定义了 PEFT 的潜力——从省钱技巧变成个性化模型的基石,做大规模模型部署和个性化服务的团队值得关注,尤其是那些需要为每个用户维护独立模型状态的场景。
本文重新审视参数高效微调(PEFT)的角色,提出将其视为在强大基础模型上附加的持久本地状态,而非仅作为全参数微调的廉价替代。研究围绕三个扩展维度展开:向上扩展(更强的共享先验使小适配器更有用)、向下扩展(研究适配器的最小可靠尺寸)以及向外扩展(大量持久适配实例共存)。MinT 基础设施示例展示了如何管理适配器的身份、版本、来源、评估和服务驻留。结果表明,PEFT 可以成为持久个性化模型的紧凑载体,而不仅仅是预算有限的微调替代方案。
On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, the base model provides shared competence while adapters carry instance-specific behavior such as preferences, skills, tool habits, and memory-like updates. We organize the problem around three scaling axes: Scale Up, where stronger shared priors make small local updates more useful; Scale Down, where we study how small adapters can be while remaining reliable; and Scale Out, where many persistent adapted instances coexist. MinT provides one infrastructure example for managing adapter identity, revision, provenance, evaluation, and serving residency. Together, the results suggest that PEFT can be a compact substrate for persistent personal models rather than only a budget substitute for full fine-tuning.