产品官方一手
Amazon SageMaker HyperPod 支持模型缓存，将推理冷启动时间从数分钟缩短至秒级
Reduce inference cold starts on Amazon SageMaker HyperPod with model caching
精选理由
亚马逊 SageMaker HyperPod 新增模型缓存功能，能将推理冷启动时间从数分钟缩短到秒级，比之前版本快很多。
Amazon SageMaker HyperPod 新增模型缓存功能，通过将模型权重和容器镜像预加载到集群节点本地 NVMe 存储中，使 pod 从本地读取数据而非网络下载，从而将推理冷启动时间从数分钟缩短至秒级。
原文 · AWS Machine Learning Blog
Reduce inference cold starts on Amazon SageMaker HyperPod with model caching
Amazon SageMaker HyperPod now supports model caching for inference, which pre-loads model weights and container images onto cluster nodes so pods read from local NVMe storage instead of downloading over the network. Learn how model caching cuts cold starts from tens of minutes to seconds, how it works, and how to enable it.