ARIADNE:推理时动态适配器选择的无关路由框架

ARIADNE: Agnostic Routing for Inference-time Adapter DyNamic sElection

精选理由

这篇论文提出了ARIADNE,一个不用训练就能自动为输入选对适配器的方法,在Llama 3.2上恢复了97%的上界性能,比现有路由方式更灵活。

AI 摘要

ARIADNE是一个无需训练、与适配器无关的动态适配器选择框架。它通过计算每个适配器训练集嵌入的质心,在推理时根据无标签输入与质心的距离选择适配器。在Llama 3.2 1B Instruct上对23个NLP任务测试,恢复了97.44%的上界性能。扩展到44个任务时,平均选择准确率达到89.7%。无需访问适配器内部参数或额外训练。

原文 · arXiv cs.AI

ARIADNE: Agnostic Routing for Inference-time Adapter DyNamic sElection

The increasing deployment of parameter-efficient fine-tuning (PEFT) has led to model ecosystems in which a single backbone is paired with many task-specialized adapters. In this setting, inference-time queries often arrive without task labels, requiring the system to automatically select the most appropriate adapter from a growing and heterogeneous adapter pool. Existing routing methods either depend on access to adapter internals, such as weight decompositions or gradient-based statistics, or require additional router training, which limits scalability and portability as new adapters are added. We introduce ARIADNE, a training-free, adapter-agnostic routing framework for dynamic adapter selection at inference time. ARIADNE represents each adapter through a set of centroids computed from embeddings of its training set, capturing the data distribution associated with that adapter. Given an unlabeled input, it selects an adapter by measuring proximity to these centroids in latent space. Because routing is performed entirely in the input embedding space, ARIADNE is compatible with arbitrary PEFT methods and requires no modification to the adapters or training procedures. Primarily evaluated with Llama 3.2 1B Instruct on 23 diverse NLP tasks, ARIADNE recovers 97.44% of the upper bound performance. Scaling to 44 tasks, it achieves 89.7% average selection accuracy, without additional training or access to adapter internals.