Latest

Wrong autoscaling thresholds on GPU endpoints either burn money on idle capacity or break latency guarantees, and most teams tune them by trial and error. The Q_max and T_total measurement method gives a reproducible formula you can apply to any custom inference workload, not just TTS. MLOps and platform engineers running SageMaker endpoints should try the walkthrough to benchmark their own scaling policy.

Primary sourceAWS Machine Learning Blog@Joseph CottinghamSource

The notable point is that one config setup now covers three different training backends on Trainium, so teams can switch between them without rewriting launch scripts or breaking checkpoint compatibility. Engineers choosing between NVIDIA GPUs and AWS silicon can use this to evaluate Trainium for fine-tuning open-source models like Qwen3. The guide also shows how NKI MoE kernels fit into the same workflow.

Primary sourceAWS Machine Learning Blog@Josh LongeneckerSource

AI coding assistants raise the volume of changes per release, which multiplies the ways a large codebase can break — making crash prevention practices more urgent, not less. Mobile engineers and platform leads can compare their own rollout, monitoring and triage setups against Grab's layered pipeline. The write-up is a rare detailed look at superapp stability engineering from a Southeast Asian company operating at Google Play scale.

The three-stage structure — gather context, generate review cards, then produce video — is a reusable pattern for anyone building AI pipelines that turn documents into presentations. Developers can compare it with their own document-to-video or summarisation agents and borrow the intermediate review-card step as a quality gate. NotebookLM users gain a clearer picture of what the tool does with their sources.

宝玉的分享@宝玉Source

The real lesson is the decision framework: pick SFT when accuracy targets are clear, and add GRPO only when benchmarks show remaining headroom, avoiding wasted RL compute. Practitioners building multilingual models on their own data can replicate the workflow on SageMaker HyperPod with open weights, no dependency on a closed API.

Primary sourceAWS Machine Learning Blog@Mijanur PalashSource

Runtime discovery changes agent architecture: registries become the integration layer, and new tools can be added without redeploying the orchestrator. Developers building multi-agent systems on AWS can adopt this pattern to keep large tool catalogs maintainable, using MCP and A2A standards already gaining traction across the ecosystem.

Primary sourceAWS Machine Learning Blog@Vibhu PareekSource

The architecture shows how x509 certificate chains can bind a specific agent instance to delegated user permissions, closing the spoofing gap that plagues token-only agent stacks. Engineers building multi-agent or MCP deployments on AWS can reuse the pattern with EJBCA and OCSP directly. It also gives security teams a concrete baseline to compare against their current agent authentication design.

Primary sourceAWS Machine Learning Blog@Deepakkumar SharmaSource

The non-obvious part is that governance travels through the MCP server: an agent querying a semantic layer still respects per-user row and column security, which removes the usual fear of agents leaking data they can technically reach. Data engineers and agent builders on AWS can follow the walkthrough to combine Redshift, Aurora, Strands and AgentCore into one governed setup. It also serves as a concrete reference implementation of MCP in an enterprise BI context.

Primary sourceAWS Machine Learning Blog@Zhili ChengSource

Governance for agentic analytics is often bolted onto prompts, which is fragile and unauditable; this shows enforcement moving down into the data layer itself. Data engineers and platform teams building agents on AWS can follow the walkthrough to expose Redshift models safely through MCP. The pattern applies anywhere enterprises need row-level security for AI agents, with AtScale as the named alternative to hand-rolled semantic layers.

Primary sourceAWS Machine Learning Blog@Frank LapolitoSource

KV cache transfer addresses the hidden cost of agentic loops: re-prefilling the same context on every turn, which inflates latency and GPU spend as conversations grow. Teams running self-hosted agents on vLLM can replicate the setup on EC2 P5en instances with EFA. The reported gains also give a concrete comparison point for choosing between Qwen3-32B and GLM-5-FP8 when latency per turn matters.

Primary sourceAWS Machine Learning Blog@Sunita NadampalliSource

Shifting KV-cache transfer coordination off the critical path removes a hidden latency tax in prefill-decode disaggregation, and the 3x TTFT gain applies to any team serving LLMs at scale on vLLM. Practitioners running inference on AWS can adopt the NIXL connector on EC2 P5en with EFA networking directly from the guide. Compared with pull-based transfer, the push design needs no receiver-side polling, which is what unlocks the latency savings.

Primary sourceAWS Machine Learning Blog@Sunita NadampalliSource