ProductsOfficial

Amazon SageMaker HyperPod 推出 Global Inference Gateway,单一端点跨区域路由推理请求

Introducing Amazon SageMaker HyperPod Global Inference Gateway: One endpoint across Regions and clusters

Why we picked it

搞自部署推理的朋友可以看看:AWS 在 HyperPod 上加了个全局网关,一个端点就能跨区域调度集群,还带自动故障转移和 PrivateLink 私有连接。之前自己写跨区路由和容灾的活,现在托管了,多区域部署的团队值得评估下。

AWS 为 Amazon SageMaker HyperPod 推出 Global Inference Gateway,用户可以通过一个托管端点将推理请求路由到单个区域内或跨多个区域的 HyperPod 集群,无需为每个集群单独管理端点。

Image · AWS Machine Learning Blog
Body

该网关的核心机制包括:自动选择最佳集群来处理请求,当某个集群出现故障时自动进行故障转移,所有请求通过 SigV4 和 IAM 进行认证,并经由 AWS PrivateLink 保持私有网络传输,推理流量不经过公网。

对于在多个区域部署大规模推理集群的企业,这意味着此前需要自行搭建跨区域路由、健康检查和容灾逻辑的工作现在由 AWS 托管,运维复杂度明显降低,也与自建 Kubernetes 联邦路由等方案形成对照。

Original · AWS Machine Learning Blog

Introducing Amazon SageMaker HyperPod Global Inference Gateway: One endpoint across Regions and clusters

Amazon SageMaker HyperPod introduces the Global Inference Gateway, one managed endpoint that routes inference requests across your HyperPod clusters in one Region or across Regions. It picks the best cluster, fails over automatically, and authenticates every request with SigV4 and IAM over AWS PrivateLink.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Amazon SageMaker, HyperPod, AWS, inference deployment.

Primary source verification: published directly by AWS Machine Learning Blog with first-party authority.