Perplexity 发布 Orchestrator API:45+ 模型统一接入,KV-cache 兼容路由
Perplexity 把 45+ 个模型塞进一个 API,还解决了路由切换会毁掉 KV cache 的老问题,比如把 fable 5.2 拒答的安全任务转给 gpt-6.1 sol。做 AI 应用的开发者值得看看它和 OpenRouter 的思路差异:路由规则由你自己定。
Perplexity 推出 Orchestrator API,将 45+ 个开源权重模型和专有模型整合到单一 API 中,提供自动供应商故障转移和统一计费。官方指出通用模型路由在实践中效果不佳,原因在于路由切换会破坏 KV cache,且容易过拟合到少数基准测试上。
Orchestrator API 的设计思路不同:路由机制对 KV cache 友好,并由开发者自己决定哪些任务使用哪些模型。一个实际例子是,fable 5.2 常常拒绝处理一些良性的安全和生物学任务,因此 Perplexity 将这类任务以及拒绝之后的内容路由到 gpt-6.1 sol。在 terminal-bench 4.1 上,这种组合将完成任务数从 38 提升到 41(共 63 个任务)。
对开发者而言,这意味着可以在保留已有 KV cache 降本优势的前提下做任务级路由,而不是在延迟和成本之间被迫取舍。相比传统通用路由方案,Orchestrator 把模型选择权交给开发者,同时用统一计费和故障转移降低多供应商运维成本。
KV-cache compatible decision-model based routing on Perplexity Orchestrator API Denis Yarats @denisyarats we are launching the Orchestrator API: 45+ open-weight and proprietary models in one api, with automatic provider failover and unified billing. general model routing doesn't really work in practice: it breaks the kv cache and is usually overfit to a few benchmarks. so we designed our routing api differently: it's kv cache friendly, and you decide which models to use for which tasks. one example: fable 5.2 often refuses benign security and biology tasks, so we route those (and anything after a refusal) to gpt-6.1 sol. on terminal-bench 4.1 this goes from 38 to 41 solved tasks out of 63. 🔗 View Quoted Tweet📊 5 ⚡
This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Perplexity, Orchestrator API, fable 5.2, gpt-6.1 sol.
Source attribution: curated from Aravind Srinivas (@AravSrinivas).