Discover more about what's new at AWS with Amazon SageMaker AI Announces New observability capability For Inference Endpoints

AWS' platform is a leading cloud computing platform, offering insights into cloud infrastructure, services, and solutions for developers, businesses, and IT professionals. Through articles, whitepapers, and documentation, AWS offers insights into cloud architecture, serverless computing, and machine learning on AWS. Developers and architects can learn about AWS services like EC2, S3, Lambda, and more to build scalable, secure, and cost-effective cloud applications.

Amazon SageMaker AI has launched a new observability capability for inference endpoints, giving teams comprehensive visibility into token performance, GPU health, inference component placement, and autoscaling behavior. A pre-built SageMaker AI Insights dashboard in Amazon CloudWatch surfaces token latency, GPU utilization, inference component copy counts, scaling events, and cold start breakdowns in a single view. OpenTelemetry-native metrics are published automatically with no instrumentation required. Teams using Grafana can connect via a regional PromQL endpoint and import a pre-configured dashboard template. The feature is available across 17 AWS regions.