用 NVIDIA NeMo Switchyard 在 Amazon Bedrock 上按任务路由编程 Agent 请求
Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard
编程 Agent 跑任务，真正该看的指标是每解决一个任务花多少钱，不是每 token 单价。这篇教程手把手用 AWS CDK 部署 LiteLLM 加 NVIDIA NeMo Switchyard，把请求路由到 Bedrock 上的不同模型，再和单模型基线比成本。自己搭 Agent 团队、想在强模型和便宜模型之间省钱的开发者可以直接照着抄。
这篇教程教你用 AWS CDK 在 Amazon Bedrock 上部署 LiteLLM，并接入 NVIDIA NeMo Switchyard 路由插件，把编程 Agent 的请求动态路由到合适的模型。
文中构建一个路由模型组（routed model group），并与单模型基线逐一对比。评估指标是每解决一个任务的成本（cost per solved task），而不是单纯看每 token 的价格，更贴近编程 Agent 实际跑任务的开销。
整套方案以 IaC 方式部署，方便团队在自己的 AWS 环境复现并按自身任务集量化路由带来的成本收益。
Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard
Coding agents work in tasks, and the number that matters is cost per solved task, not price per token. In this post, you use the AWS CDK to deploy LiteLLM with the NVIDIA NeMo Switchyard routing plugin on Amazon Bedrock, then measure cost per solved task for a routed model group against single-model baselines.
This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Amazon Bedrock, NVIDIA NeMo Switchyard, LiteLLM, coding assistants.
Primary source verification: published directly by AWS Machine Learning Blog with first-party authority.