Coinbase创始人Brian Armstrong分享了一套实际操作方案：用更便宜的默认模型、优化缓存和路由，能把AI成本砍半。开源模型GLM 5.2和Kimi 2.7是主角，缓存命中率从5%跳到60%。
Brian Armstrong在推文中分享了Coinbase控制AI成本的实践。他提到，通过将默认模型切换到开源模型如GLM 5.2和Kimi 2.7，91%的员工从未触及使用上限。通过改进缓存，LibreChat的缓存命中率从5%提升到60%。这些措施使AI支出减少近一半，同时token用量仍在增长。他还强调路由优化和精简上下文的重要性。
the future of AI is multi-model (including a majority of open-source ones provided by @huggingface o...
the future of AI is multi-model (including a majority of open-source ones provided by @huggingface of course!)! Brian Armstrong @brian_armstrong How to keep AI spend flat while token usage grows exponentially: Not with friction and spend alerts. With better defaults, routing, and caching. Better Defaults (not Usage Caps) – Engineers can choose any model they want, but defaults matter. We’re experimenting with defaulting to open weight models like GLM 5.2 and Kimi 2.7 through our LLM gateway, while still encouraging engineers to choose the right model for the task. 91% of our employees were never hitting their usage caps, so instead of lowering caps and driving up alerts, we're moving to cheaper defaults. Note that code reviews use a diversity of models, so they can check each other's work. Better Routing – In our custom harnesses, we preprocess prompts and route to the best model for the job, considering cache hits and model pricing. For instance, you may want a frontier model for planning, but not for execution where they can be overkill. Ultimately, humans shouldn't be choosing models - AI can automate this task. Better Caching – Cache misses are the easiest way to drive your cost up. All of our requests are cache aware, so we’re reusing a warm cache wherever possible. For example, our cache hit rate went from 5% → 60% in LibreChat once properly implemented. Keep Context Lean – Start fresh sessions when switching tasks. Scope file context narrowly. Disconnect unused tools. Don't just compact. The goal isn't fewer tokens used, it's fewer tokens wasted. Better Visibility – Our engineers can use as many tokens as they want, from whatever model they want, but we’ve made usage visible – and the more you spend on AI, the more impact we expect. The goal isn't to suppress usage. It's to build the infrastructure that makes exponential growth sustainable. Putting this into practice has cut our AI spend nearly in half, while our token usage continues to grow. 🔗 View Quoted Tweet 💬 7 🔄 4 ❤️ 22 👀 3423 📊 8 ⚡