这论文把“不确定就多算”的LoRA路由换成了信息价值定价,能证明预算花在哪风险降最多,还带回答弃权判断。
VI-MoLE提出将混合LoRA专家路由视为认证信息价值分配问题。该方法在验证集上为每个专家前缀后的反事实风险输出同时有效的上界证书,并依据单位成本的边际风险降低量分配全局适配器预算。与仅依赖不确定性的动态路由不同,VI-MoLE能区分可恢复风险与残余歧义。论文在匹配计算量准确率、证书覆盖率、风险-覆盖率、分布偏移和尾延迟上与固定及动态MoE-LoRA路由对比。实验设置来自arXiv 2608.02528v1。
Uncertainty Is Not Enough: Value-of-Information Routing for Mixtures of LoRA Experts
Mixtures of low-rank adaptation experts increase parameter-efficient capacity by routing each input through a subset of adapters. Recent dynamic routers activate more experts when the router or prediction is uncertain. This rule silently equates uncertainty with useful additional computation: an uncertain example may contain complementary, unqueried expert evidence, but it may instead remain ambiguous after every expert agrees. We formulate routing as certified value-of-information allocation. VI-MoLE learns the counterfactual risk remaining after each expert prefix, converts these predictions into simultaneous upper-risk certificates on held-out calibration data, and spends a global adapter budget on the token--layer action with the largest certified marginal risk reduction per unit cost. A terminal certificate then decides whether to answer or abstain. Unlike an uncertainty gate, this procedure distinguishes present ambiguity from recoverable and residual risk. We prove simultaneous certificate validity, optimal greedy allocation under diminishing certified gains, and allocation regret under value-estimation error. The evaluation protocol tests matched-compute accuracy, certificate coverage, risk--coverage, distribution shift, and tail latency against fixed and dynamic MoE-LoRA routers.