精选理由
OpenAI推出提示缓存技术，成本降90%，但每秒仅15次请求，UnifyGTM通过自定义路由实现95%命中率
OpenAI的提示缓存技术可将请求成本降低90%。该缓存键每秒最多处理约15个请求。UnifyGTM围绕这一限制构建了自己的路由系统。该系统实现了接近95%的缓存命中率。
原文 · LangChain
OpenAI's prompt cache makes a request 90% cheaper, but the cache key tops out around 15 requests per...
OpenAI's prompt cache makes a request 90% cheaper, but the cache key tops out around 15 requests per second. @HeggieConnor on how @unifygtm built its own routing around that limit, landing them close to a 95% cache hit rate. Your browser does not support the video tag. 🔗 View on Twitter 💬 3 🔄 0 ❤️ 5 👀 2113 📊 4 ⚡