--- title: Amazon SageMaker HyperPod enhances support for Ray description: Amazon SageMaker HyperPod now offers deeper Ray integration, adding built-in observability, resilient training, accelerated inference, and managed development... canonical: https://daily.dev/posts/amazon-sagemaker-hyperpod-enhances-support-for-ray-chlnsklwp twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Amazon SageMaker HyperPod enhances support for Ray | daily.dev og:description: Amazon SageMaker HyperPod now offers deeper Ray integration, adding built-in observability, resilient training, accelerated inference, and managed development... og:url: https://daily.dev/posts/amazon-sagemaker-hyperpod-enhances-support-for-ray-chlnsklwp og:image: https://api.daily.dev/og/posts/CHLnsKlwp.png og:image:alt: Amazon SageMaker HyperPod enhances support for Ray og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Amazon SageMaker HyperPod enhances support for Ray **[AWS](https://daily.dev/sources/aws)** · 2 min read · 0 upvotes · 0 comments ## Summary Amazon SageMaker HyperPod now offers deeper Ray integration, adding built-in observability, resilient training, accelerated inference, and managed development environments. Data scientists can create and manage Ray clusters from SageMaker Studio and attach interactive IDEs directly to running clusters instead of resubmitting jobs. New features include Grafana dashboards fed by Managed Prometheus, one-click Ray Dashboard access, node auto recovery and hung job detection, tiered checkpointing, task governance for compute quotas and preemption, and a tiered KV cache for Ray Serve to reduce time to first token. Open-source Ray code runs unchanged, and the feature is available for HyperPod clusters orchestrated by Amazon EKS in supported AWS Regions. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What new Ray features does Amazon SageMaker HyperPod support now? HyperPod adds built-in observability, resilient training, accelerated inference, and managed development environments for Ray. Data scientists can create and manage Ray clusters from a web interface in SageMaker Studio, attach JupyterLab or a local IDE for interactive iteration, and get Grafana dashboards backed by Amazon Managed Service for Prometheus plus one-click Ray Dashboard access. _daily.dev helps ML engineers track platform updates like this before wiring them into a training pipeline._ ### How does SageMaker HyperPod keep long Ray training runs from failing on GPU errors? HyperPod uses node auto recovery and hung job detection to handle GPU faults, job hangs, loss spikes, and degraded throughput automatically. Tiered checkpointing restores state from cluster memory to maximize goodput, while task governance applies quotas, priorities, and preemption to improve compute utilization across long-running training jobs. _Engineers debugging flaky distributed training jobs can follow infra changes like this on daily.dev._ ### How does Ray Serve on SageMaker HyperPod speed up inference? A tiered KV cache reuses cached prefixes to reduce time to first token for Ray Serve deployments on HyperPod. It also lets you deploy Amazon SageMaker JumpStart models directly, and the feature works with open-source Ray code unchanged on HyperPod clusters orchestrated by Amazon EKS. _Teams evaluating inference stacks can compare options like this on daily.dev before committing._ ## Similar posts on daily.dev - [Amazon SageMaker HyperPod now supports data capture for inference workloads](https://daily.dev/posts/amazon-sagemaker-hyperpod-now-supports-data-capture-for-inference-workloads-dk42gearn) · AWS · 0 upvotes · 0 comments - [From notebooks to nodes: Architecting production-ready AI infrastructure](https://daily.dev/posts/from-notebooks-to-nodes-architecting-production-ready-ai-infrastructure-orotpqycd) · The New Stack · 1 upvotes · 0 comments - [SageMaker HyperPod now supports Managed tiered KV cache and intelligent routing](https://daily.dev/posts/sagemaker-hyperpod-now-supports-managed-tiered-kv-cache-and-intelligent-routing-zlicgs1ml) · AWS · 2 upvotes · 0 comments - [Ray on GKE: New features for AI scheduling and scaling](https://daily.dev/posts/ray-on-gke-new-features-for-ai-scheduling-and-scaling-xwquzmlcf) · Google Cloud · 0 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#aws](https://daily.dev/tags/aws), [#kubernetes](https://daily.dev/tags/kubernetes), [#gpu](https://daily.dev/tags/gpu) [View this post on daily.dev](https://daily.dev/posts/amazon-sagemaker-hyperpod-enhances-support-for-ray-chlnsklwp) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Amazon SageMaker HyperPod enhances support for Ray","url":"https://daily.dev/posts/amazon-sagemaker-hyperpod-enhances-support-for-ray-chlnsklwp","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/amazon-sagemaker-hyperpod-enhances-support-for-ray-chlnsklwp"},"datePublished":"2026-08-24T18:06:20.326Z","dateModified":"2026-08-24T18:06:42.696Z","description":"Amazon SageMaker HyperPod now offers deeper Ray integration, adding built-in observability, resilient training, accelerated inference, and managed development...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2587ee7271b19b73b7c9d67f739e4290?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2587ee7271b19b73b7c9d67f739e4290?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"AWS","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"AWS","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/aws","url":"https://daily.dev/sources/aws"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/amazon-sagemaker-hyperpod-enhances-support-for-ray-chlnsklwp","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,aws,kubernetes,gpu","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"AWS","item":"https://daily.dev/sources/aws"},{"@type":"ListItem","position":3,"name":"Amazon SageMaker HyperPod enhances support for Ray"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/amazon-sagemaker-hyperpod-enhances-support-for-ray-chlnsklwp#faq","mainEntity":[{"@type":"Question","name":"What new Ray features does Amazon SageMaker HyperPod support now?","acceptedAnswer":{"@type":"Answer","text":"HyperPod adds built-in observability, resilient training, accelerated inference, and managed development environments for Ray. Data scientists can create and manage Ray clusters from a web interface in SageMaker Studio, attach JupyterLab or a local IDE for interactive iteration, and get Grafana dashboards backed by Amazon Managed Service for Prometheus plus one-click Ray Dashboard access. daily.dev helps ML engineers track platform updates like this before wiring them into a training pipeline."}},{"@type":"Question","name":"How does SageMaker HyperPod keep long Ray training runs from failing on GPU errors?","acceptedAnswer":{"@type":"Answer","text":"HyperPod uses node auto recovery and hung job detection to handle GPU faults, job hangs, loss spikes, and degraded throughput automatically. Tiered checkpointing restores state from cluster memory to maximize goodput, while task governance applies quotas, priorities, and preemption to improve compute utilization across long-running training jobs. Engineers debugging flaky distributed training jobs can follow infra changes like this on daily.dev."}},{"@type":"Question","name":"How does Ray Serve on SageMaker HyperPod speed up inference?","acceptedAnswer":{"@type":"Answer","text":"A tiered KV cache reuses cached prefixes to reduce time to first token for Ray Serve deployments on HyperPod. It also lets you deploy Amazon SageMaker JumpStart models directly, and the feature works with open-source Ray code unchanged on HyperPod clusters orchestrated by Amazon EKS. Teams evaluating inference stacks can compare options like this on daily.dev before committing."}}]} ```