ProductsOfficial

Salesforce 用 Amazon SageMaker AI 跨区域推理路由实现全球模型服务

How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI

Why we picked it

Salesforce 把 SageMaker AI 的跨区域路由用在了生产里,容量不够时自动切区域,还能看到每个请求被谁服务、为什么被转发。做多云或多区域 LLM 服务部署的工程团队可以参考这套架构。

Salesforce 在 Amazon SageMaker AI 中启用跨区域推理路由,把分散在多个区域和账户中的推理容量整合成一个统一的服务入口。当某个区域的容量不足时,路由器会自动把请求转发到其他可用区域,避免单一区域的资源瓶颈影响在线服务。

Image · AWS Machine Learning Blog
Body

路由机制在转发的同时保留 Salesforce 对数据驻留(data residency)的合规要求,不把数据送到承诺范围之外的区域。每次请求都可以查到它实际被哪个区域服务、被转发的原因,可观测性贯穿整条调用链。

对同时在多个 AWS 区域和账户部署大模型服务的企业来说,这种做法把区域碎片化容量的利用率提上来,也降低了为高峰流量在各区域重复预留资源的成本。

Original · AWS Machine Learning Blog

How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI

Learn how Salesforce uses the cross-Region inference router in Amazon SageMaker AI to turn fragmented Regional capacity into reliable model serving. Salesforce adds capacity securely across Regions and accounts while keeping its data residency commitments, and sees per request where each call was served and why.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Amazon SageMaker AI, Salesforce, AWS, model routing.

Primary source verification: published directly by AWS Machine Learning Blog with first-party authority.