hwchase17聊了模型路由和合议制的区别，吐槽路由被夸大，推荐先做成本控制。适合想了解多模型调度策略的人。
LangChain创始人hwchase17对比了模型路由（route to one best model）和模型合议制（pass to many models并聚合响应），认为前者主要为了成本，后者提升前沿性能。他引用@OpenRouter的Fusion和@SakanaAILabs的fugu作为合议制案例，并提到@nlarusstone将其用于棘手生物问题。作者对模型路由效果存疑，认为prompt caching和用户反馈缺失是问题，更倾向先实施成本控制（如LangChain的按模型设置支出上限）。Dax补充了用户需培养模型感知的观点。
have been thinking a bunch about model routing and related things current thoughts here, would love...
have been thinking a bunch about model routing and related things current thoughts here, would love feedback: 1/ there is a difference between "model routing" and "model council" "model routing" = route to one best model. i think only real reason to do this is cost "model council" = pass to many models at the same time, then aggregate responses. this is done to push frontier performance 2/ "model council" seems to be having a moment recent. @OpenRouter did their Fusion thing, @SakanaAILabs did fugu I do think there are use cases where this can be useful - @nlarusstone highlights how they use it for tough bio problems and the most recent "max agency" i'm unsure how you would use it to drive a main agent, but as a "tool" makes sense. similar to how anthropic's advisor tool 3/ model routing in theory sounds nice and im probably a bit more bullish that dax. I dont think expecting ALL users to be on top of which model to use when is feasible. (maybe for coding its more feasible, im thinking broader tho). prompt caching is a real concern. maybe you can heavily bias towards not switching once its been routed once initially? 4/ main reason for model routing is cost. but there are probably other (simpler) levers to pull first. eg putting spending caps, maybe on a per model basis (you get X dollars for opus, then need to use sonnet) at langchain we are doing (4) first (basic cost controls/policies: langchain.com/blog/how-we-ma… ) but considering routing and other options dax @thdxr i have low conviction on model routers - very open to changing my mind but this is a snapshot of my current thoughts - i don't think it's good to not be aware of what model you're using. coding with LLMs is a skill you develop and getting a feel for models is part of that - people (at scale) don't have this skill right now which is why a lot of companies are complaining that people are using expensive models for dumb things. a model router promises to solve this without the user having to do anything but i think the issue is missing feedback loops to the user. id rather we figure out how to help users get smarter - i dont even know how much you can model route when factoring in things like prompt cache. only so much you can do - their effectiveness is a bit exaggerated by the same dynamic that's impacting everything AI. so many companies desperately searching for opportunities and trying anything. model routing is the one thing models labs cannot do so everyone is jumping on it 🔗 View Quoted Tweet 💬 5 🔄 1 ❤️ 6 👀 2820 📊 5 ⚡