Jerry Liu 说路由优化放 harness 层比网关层划算，能更好平衡准确率和成本，跑智能体任务的人值得看。
Jerry Liu在X上表示，LLM路由优化应放在harness层而不是网关层。每个端到端任务都需要不同的LLM组合与harness逻辑，才能优化准确率和成本。仅在网关层优化LLM组合，会丢失harness中的上下文，只优化到LLM补全层。而在harness层优化LLM选择，可以在智能体循环中预先做出最优决策。
It makes sense to optimize model routing at the harness layer instead of the gateway layer if you wa...
It makes sense to optimize model routing at the harness layer instead of the gateway layer if you want to hillclimb on accuracy/cost for any e2e task. Every task is solved by a combination of a model mixture and agent harness. Every task requires a different mixture of models (+harness logic) to be at the pareto frontier of accuracy and cost. * If you only optimize the model mixture at the gateway layer, you lose the broader context encoded in the harness and only optimize at the LLM completion layer. * If you optimize the model mixture at the harness layer, you can make the "optimal" model choices a priori and while in the agent loop For any given task, the model and harness are probably co-optimized together, so that the model mixture can only exist with this specific harness shape and vice versa Abhay Singhal @_AbhaySinghal x.com/i/article/2087… 🔗 View Quoted Tweet 💬 2 🔄 0 ❤️ 2 👀 948 📊 2 ⚡