如果你在多个大模型间做路由，这个框架能直接优化成本和效果，不用先预测质量，实测在10个基准上超过现有方法，还能兼容新模型。
RLCascadeRouter提出了一种无需质量估计器的级联路由框架，将路由问题建模为马尔可夫决策过程，动作包括停止和模型选择。该方法直接优化性能-成本目标，避免了传统预测-优化流程中预测误差与决策损失不一致的问题。在包含13个LLM的10个LLMRouterBench基准上，RLCascadeRouter优于强基线，实现了更优的性能-成本权衡。它还能无需重训练地纳入未见模型，消融研究验证了策略组件的有效性。
RLCascadeRouter: Quality-Estimator-Free Cascade Routing via Reinforcement Learning
The growing ecosystem of large language models (LLMs) offers huge potential to optimize performance-cost trade-offs. However, their heterogeneous capabilities and inference costs make efficiently routing queries a significant challenge. Existing paradigms are inflexible: one-shot routers commit before observing responses, whereas conventional cascades stop adaptively but follow a fixed model order. Cascade routing removes both restrictions by reconsidering whether to stop or invoke another model after each response. Current methods use a predict-then-optimize pipeline estimating response quality and future model utility. However, prediction loss for quality or utility is not equivalent to routing-decision loss. A lower prediction error does not necessarily yield a better action; a small boundary-crossing error can reverse a ``stop'' or model-selection decision. Therefore, we propose RLCascadeRouter, a quality-estimator-free framework that formulates cascade routing as a Markov decision process with actions comprising ``stop'' and model selection. It uses trajectory returns and advantages to directly optimize the performance-cost objective. Its Cascade Policy Network models candidate complementarity for model selection and remaining-action value for stopping, eliminating independent post-hoc response-quality estimators. Evaluated across ten LLMRouterBench benchmarks with thirteen LLMs, RLCascadeRouter outperforms strong baselines and achieves superior performance-cost trade-offs. It incorporates unseen models without retraining, and ablation studies validate both policy components.