This 2026 update increased the per-job shuffle limit from 200 GB to 1 TB, enabling Apache Spark workloads with large-scale data shuffling, such as multi-terabyte joins and complex aggregations, to run without storage limitations. Spill support was also added to offload data to disk during memory-intensive operations, raising job reliability and success rates.
These enhancements facilitate migration of demanding production workloads to EMR Serverless by eliminating previous storage constraints. The feature is available from Amazon EMR versions 7.14 and Spark 8.1 across 18 AWS Regions where EMR Serverless is offered.
Source & further reading
Serverless Storage on Amazon EMR Serverless now supports terabyte-scale shuffle ↗Source published 2026-10-01. This report was prepared with AI assistance from the linked primary-source material. Claims and limitations should be checked against the original announcement or research.