模型路由优化：Harness层优于网关层
Jerry Liu 说路由优化放 harness 层比网关层划算，能更好平衡准确率和成本，跑智能体任务的人值得看。
Jerry Liu在X上表示，LLM路由优化应放在harness层而不是网关层。每个端到端任务都需要不同的LLM组合与harness逻辑，才能优化准确率和成本。仅在网关层优化LLM组合，会丢失harness中的上下文，只优化到LLM补全层。而在harness层优化LLM选择，可以在智能体循环中预先做出最优决策。
It makes sense to optimize model routing at the harness layer instead of the gateway layer if you want to hillclimb on accuracy/cost for any e2e task. Every task is solved by a combination of a model mixture and agent harness. Every task requires a different mixture of models (+harness logic) to be at the pareto frontier of accuracy and cost. * If you only optimize the model mixture at the gateway layer, you lose the broader context encoded in the harness and only optimize at the LLM completion layer. * If you optimize the model mixture at the harness layer, you can make the "optimal" model choices a priori and while in the agent loop For any given task, the model and harness are probably co-optimized together, so that the model mixture can only exist with this specific harness shape and vice versa Abhay Singhal @_AbhaySinghal x.com/i/article/2087… 🔗 View Quoted Tweet 💬 2 🔄 0 ❤️ 2 👀 948 📊 2 ⚡
