#Amazon EKS

Disaggregated prefill and decode across nodes is becoming the default architecture for serving large models efficiently, and AWS now packages it in a container you can run without building the stack yourself. Engineers running inference on EKS can use this to serve models that exceed a single node's GPU capacity with an OpenAI-compatible API. It competes directly with self-assembled vLLM or Ray Serve setups that require far more glue code.

Primary sourceAWS Machine Learning Blog@Ananth RaghavendraSource

A 40% rollout throughput gain comes from pairing EFA with DeepEP, meaning the bottleneck for MoE RLHF was inter-node communication rather than GPU compute. Teams running GRPO or RLHF pipelines on Kubernetes can replicate the EKS + EFA + S3 setup directly. The write-up also serves as a reference for comparing managed EKS clusters against self-built multi-node RDMA training stacks.

Primary sourceAWS Machine Learning Blog@Ashvin NihalaniSource

Available on AWS in ASEAN, this integration simplifies distributed ML workflows without requiring Ray expertise.

Primary sourceAWS Machine Learning Blog@Nilesh PSSource

AWS出了个新插件，能在你已有的EKS集群上直接跑JupyterLab和Code Editor，不用另起炉灶，还支持VS Code远程连。

Primary sourceAWS Machine Learning Blog@Rajat JainSource