Skip to content

[SPARK-60864] Publish Apache Spark 4.3.0 to docker registry - #143

Closed
HeartSaVioR wants to merge 1 commit into
apache:masterfrom
HeartSaVioR:spark-4.3.0-docker
Closed

HeartSaVioR wants to merge 1 commit into
apache:masterfrom
HeartSaVioR:spark-4.3.0-docker

Conversation

@HeartSaVioR

@HeartSaVioR HeartSaVioR commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Publish Apache Spark 4.3.0 to the Docker registry:

  • Generate Scala 2.13 images for Java 17, 21, and 25, with Scala, Python, R, and Python/R variants.
  • Register the 4.3.0 release signing key and add the build/test workflow.
  • Add 4.3.0 to the publish/test workflows and make it the default version.
  • Add versioned image tags and move the floating tags, including latest, from 4.2.1 to 4.3.0.

The Dockerfiles and entrypoints are generated using ./add-dockerfiles.sh 4.3.0.

Why are the changes needed?

Apache Spark 4.3.0 has been released:

This follows the release publication changes in #141 for Spark 4.2.1.

Does this PR introduce any user-facing change?

Yes. Adds Spark 4.3.0 image tags and updates the floating tags to use 4.3.0.

How was this patch tested?

  • Generated all 12 image variants with ./add-dockerfiles.sh 4.3.0. Apart from the version and the GPG key, they are identical to the 4.2.1 variants.
  • Checked the release signature's issuer fingerprint (FD3E84942E5E6106235A1D25BD356A9F8740E4FF) against Apache Spark KEYS and the public keyserver.
  • Compared the full diff with [SPARK-60639] Publish Apache Spark 4.2.1 to docker registry #141 after normalizing version strings. Only the GPG key and surrounding context lines differ.
  • Validated JSON/YAML, unique image tags, preservation of existing versioned tags, floating tag ownership (all on 4.3.0), and workflow matrices/defaults.
  • Checked all three generated entrypoints with bash -n and verified their executable permissions.
  • Ran the manifest tag generator and git diff --check.
  • Built 4.3.0/scala2.13-java17-ubuntu locally with docker build (including GPG and SHA-512 verification of the release tarball). Then ran:
    • spark-submit --version: reports Spark 4.3.0, Scala 2.13.18, OpenJDK 17.0.21.
    • run-example SparkPi 10: prints Pi is roughly 3.1409....

The other image builds and the standalone/Kubernetes integration tests will run through the new Java 17/21/25 CI matrix.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5.5)

Co-authored-by: Isaac <no-reply@databricks.com>
@HeartSaVioR

Copy link
Copy Markdown
Contributor Author

Merge Summary:

Posted by merge_spark_docker_pr.py

yosifkit pushed a commit to docker-library/official-images that referenced this pull request Nov 9, 2026
Add the Spark 4.3.0 images from apache/spark-docker#143 and move the
rolling tags from 4.2.1 to 4.3.0.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants