Amazon SageMaker AI adds concurrency sweeps for endpoint right-sizing
The CreateAIBenchmarkJob API is available to any SageMaker AI customer, so teams hosting open-source models like Llama can cut inference costs by matching instance types to real traffic instead of over-provisioning.