AWS Glue Interactive Sessions added support for Apache Spark Connect, enabling users to develop and run Spark applications from their preferred environments such as Amazon SageMaker Unified Studio, Jupyter, or Visual Studio Code. This integration allows serverless execution without managing clusters and supports workflows like ad hoc data exploration and incremental PySpark job development.
Spark Connect’s thin client architecture decouples client applications from the Spark execution environment, isolating client dependencies from server runtime, which AWS claims improves stability and upgrade simplicity. Observability features include real-time session monitoring via Spark UI and history tracking, with management through AWS APIs, CLI, or SDK, available in multiple global AWS regions.
Source & further reading
AWS Glue Interactive Sessions now support Spark Connect for interactive workloads ↗Source published 2026-06-17. This report was prepared with AI assistance from the linked primary-source material. Claims and limitations should be checked against the original announcement or research.