CacheRouter通过双路径设计优化了LLM系统中的工具使用，自动化工具注册和运行时更新，显著提高缓存命中率，降低输入成本，值得一试。
本文提出CacheRouter，一种双路径路由设计，将工具选择和交付分配到不同的通道。主模型始终看到一组固定的核心工具，其他工具通过独立的路由通道选择和执行。工具注册自动化，支持运行时更新。在55个功能查询和30轮对话中测试，缓存命中率达到90.99%和95.2%，在DeepSeek定价下，缓存命中输入令牌成本约为缓存未命中令牌的1/30。
CacheRouter: A Dual-Path Tool Routing Architecture with Cache-Preserving Main-Model Isolation for Long-Tail Tool Discovery
Tool use in LLM systems faces a structural trade-off. Progressive disclosure keeps the prompt small by showing only the tools relevant to the current task, while prompt caching rewards a request prefix that stays fixed across calls; every change to the visible tool list invalidates the cached prefix. This paper treats the trade-off as a problem of request architecture and proposes a dual-path routing design that assigns tool selection and tool delivery to separate channels. The main model always sees a small, fixed set of core tools, so the head of its request is unchanged across calls; all other tools are reached through an independent routing channel, in which a router sub-model searches the full tool list, selects one tool, executes it, and returns the result. Tool registration is automated from source code and supports runtime updates, so the tool set can grow without modifying the main model's request prefix. The design generalizes progressive disclosure: capabilities are disclosed through the routing channel, and the main model's prefix stays stable. A prototype implementation was exercised on 55 functional queries and a 30-turn dialogue; token-level cache hit rates reached 90.99% and 95.2%, cutting input cost to about 12.0% and 8.0% of a no-cache baseline under DeepSeek's pricing, where cache-hit input tokens cost roughly 1/30 of cache-miss tokens.