研究实测:code mode 比原生 MCP 省 70-90% token,路由层反而不划算
Beyond Plain MCP: Do Decision Models and Code Mode Make Tool Use Cheaper and Faster?
做 MCP 工具调用的开发者该看看:code mode 直接省 70-90% token,stub 里写清返回值形状能让弱模型成功率涨最多 39.5 个点,比加路由层有效多了。harness 和 4B 选择模型都开源了,想省成本的可以照着搭。
arXiv 上一篇预注册评测比较了 MCP 网关的两种优化范式:让智能体直接对着工具 stub 写脚本的 code mode,以及在前面预选工具子集的决策路由层。实验在 12 个服务器、最多 1,958 个干扰工具的环境下进行,使用 Qwen、Gemma、Nemotron 三个开源权重智能体,并在密封测试集上验证结果。
结果显示 code mode 明显优于原生 MCP:token 消耗降低 70-90%,任务效用同时提升。作者发现一个关键变量——在 code mode 的 stub 中声明返回值形状提示,能让较弱的智能体成功率最高波动 39.5 个百分点,这个效应比加一个路由层还大。而核心假设未获证实:在 code mode 之前加决策层,只对较弱智能体带来统计上不显著的改善,对最强智能体反而略有成本。
团队公开了完整评测 harness、一个微调的 4B 工具选择模型以及全部评测轨迹,方便社区在本地隐私部署和智能体架构上复现实验。
Beyond Plain MCP: Do Decision Models and Code Mode Make Tool Use Cheaper and Faster?
Model Context Protocol (MCP) gateways manage tool catalogs for language model agents using two primary paradigms: code mode where agents write scripts against tool stubs and decision layers which are cheap routers that preselect tool subsets. Using a controlled harness across 12 servers, up to 1,958 distractor tools, and three open-weight agents including Qwen, Gemma, and Nemotron, we present a preregistered evaluation on a sealed test split to determine when each approach succeeds. Code mode substantially outperforms plain MCP, cutting token usage by 70 to 90 percent while significantly boosting utility. Declaring return-shape hints in code mode stubs is a first-order variable, swinging success by up to 39.5 points for lower-performing agents, an effect larger than adding a router. The central hypothesis that a decision layer in front of code mode beats code mode alone is not confirmed, showing minor, statistically non-significant improvements for weaker agents and a slight cost for the strongest. We publicly release our harness, a finetuned 4B tool selection model, and all evaluation traces to support robust community experimentation on local AI privacy and agent architectures.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include MCP and tools, MCP, Qwen, Gemma.
Source attribution: curated from arXiv cs.AI (@Abhishek Rai).