Discover more about what's new at AWS with Run Interactive Workloads on Amazon EMR Serverless with Spark Connect

AWS' platform is a leading cloud computing platform, offering insights into cloud infrastructure, services, and solutions for developers, businesses, and IT professionals. Through articles, whitepapers, and documentation, AWS offers insights into cloud architecture, serverless computing, and machine learning on AWS. Developers and architects can learn about AWS services like EC2, S3, Lambda, and more to build scalable, secure, and cost-effective cloud applications.

Amazon EMR Serverless now supports interactive sessions via Spark Connect, allowing developers to run Apache Spark workloads from notebooks (Jupyter, VS Code) and IDEs, including Amazon SageMaker Unified Studio. The client-server architecture of Spark Connect decouples the application client from the Spark driver, enabling persistent Spark contexts across cells, local Python blended with remote Spark operations, and iterative PySpark development. Observability features include real-time Spark UI monitoring, Spark History Server, and session management via the EMR console or API/CLI/SDK. Available with EMR release 7.13 in all supported AWS regions.