Apache Spark
Tag2.3K stories
Apache Spark news and updates covering a distributed engine for large-scale data processing. Readers can learn about the DataFrame and SQL APIs, execution planning and shuffles, structured streaming, tuning memory and partitions, running on Kubernetes or managed services, and lighter alternatives.
Amazon EMR 7.1 now supports Trino 435, Python 3.11Amazon EMR Serverless announces detailed performance monitoring of Apache Spark jobs with Amazon Managed Service for PrometheusUnity Catalog Lakeguard: Industry-first and only data governance for multi-user Apache™ Spark clustersFunctional Elegance: Making Spark Applications Cleaner with the Cats LibraryHow Data Cloud Processes One Quadrillion Records MonthlySubqueries and CTEs in Spark: Enhancing Data Analysis and ManipulationIntroduction to Apache Spark | Part 2What is Apache Spark? The big data platform that crushed HadoopUpstage AI Introduces Dataverse for Addressing Challenges in Data Processing for Large Language ModelsUser-defined aggregation functions in Spark
