Apache Spark 4.3 ships in Databricks Runtime 19
The quarterly release cadence is the quiet headline: Spark teams can expect completed features to land sooner instead of waiting on long release cycles. Data engineers running real-time pipelines should evaluate the new streaming aggregation and transformWithState APIs against existing Flink or Kafka Streams setups. Availability comes via Databricks Runtime 19, so upgrades happen through the platform rather than a standalone download.
Databricks released Apache Spark 4.3 as part of Databricks Runtime 19. The release adds streaming aggregation and transformWithState to Real-Time Mode, SCD Type 2 Auto CDC for history-preserving pipelines, and Project Feather for faster small-data iteration. Spark Connect also gains expanded capabilities so more applications can reach Spark. Spark 4.3 marks the start of a quarterly feature-release cadence.
Introducing Apache Spark® 4.3, now available in Databricks Runtime 19!
The release brings major new capabilities across real-time processing, history-preserving pipelines, small-data performance, Spark Connect, and SQL.
What’s new:
- Bring state to Real-Time Mode with streaming aggregation and transformWithState - Preserve history with SCD Type 2 Auto CDC - Iterate faster on small data with Project Feather - Reach Spark from more applications with expanded Spark Connect capabilities
It also begins a new quarterly feature-release cadence, bringing completed improvements to users sooner.
Get started: https://t.co/FF22P4x06r
