Add multi-arch (amd64+arm64) buildx support for alpine Docker image#414
Draft
ihalatci wants to merge 1 commit into
Draft
Add multi-arch (amd64+arm64) buildx support for alpine Docker image#414ihalatci wants to merge 1 commit into
ihalatci wants to merge 1 commit into
Conversation
Replace the single-arch Python Docker SDK build/tag/push path in publish_docker.py with `docker buildx build` subprocess calls so the alpine Citus image (built from source, no arch-specific packages) can produce linux/amd64 + linux/arm64 manifests. - get_platforms() gates alpine multi-arch behind DOCKER_BUILD_MULTI_ARCH (default OFF -> amd64 only); every other image stays amd64-only. - run_buildx_build() makes one buildx call per image: --push when publishing, --load for local single-arch parity, build-only for multi-arch non-publish (validates arm64 compile, output discarded). - Tagged flow passes the full semver cascade as multiple -t flags in a single build+push; removed dead SDK clients and flush_logs(). - Relax alpine tagged test to rely on buildx check=True. - Add QEMU + buildx setup to publish-docker-image-tests.yml (var unset, keeping tools CI amd64-fast). Refs citusdata/citus#8612 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
linux/arm64build capability to the Citus alpine Docker image by replacing the single-arch Python Docker SDK build/tag/push path inpackaging_automation/publish_docker.pywithdocker buildx buildsubprocess calls.Refs citusdata/citus#8612.
Why
The base images (
postgres:18.4/postgres:18.4-alpine) already publisharm64/v8manifests. The alpine Citus image builds Citus from source (./configure && make install) with no arch-specific packages, so it is arm64-capable today. The Debian images install pinned.debpackages that don't yet exist for arm64 (handled separately), so only alpine goes multi-arch here — every other image stays amd64-only with functionally unchanged behavior.Changes
get_platforms()— gates alpine multi-arch behind env varDOCKER_BUILD_MULTI_ARCH(truthy set1/true/yes/on). Default OFF →linux/amd64for all images, preserving today's behavior. When ON, onlyalpine→linux/amd64,linux/arm64.run_buildx_build()— onedocker buildx buildcall per image:--push(multi-arch manifests can't be--loaded; must go straight to the registry),--load(parity with old local-image behavior),14,14.1,14.1.0) as multiple-tflags in a single build+push.import docker, module-level clients,flush_logs()).Companion changes (ride together to avoid a knowingly-red CI)
test_publish_tagged_docker_images_alpineto drop theimages.get(...)asserts and rely onrun_buildx_build'scheck=True(gate-agnostic; with the var ON alpine is build-only so there's no local image toget).docker/setup-qemu-action@v3+docker/setup-buildx-action@v3topublish-docker-image-tests.yml.DOCKER_BUILD_MULTI_ARCHis intentionally left unset so tools PR/unit CI stays amd64-fast; arm64 is exercised at publish time by the citusdata/docker workflows.Compatibility & follow-ups
--pushrequires a priordocker login— the citusdata/docker publish workflows already log in; buildx reads~/.docker/config.json, so ordering is compatible.citusdata/toolsrelease tag must be cut and the citusdata/docker workflows repinned (currentlycitusdata/tools@v0.8.36).Risk
Emulated arm64 alpine source-build under QEMU is slow (30–60+ min). Mitigated by the default-OFF env gate — the tools repo never pays this cost unless a workflow explicitly opts in.
Validation
Local:
py_compileOK on both Python files; grep confirms zero leftoverdocker_client/docker_api_client/flush_logs/import dockerrefs; AST check confirms all test-imported symbols present andflush_logsremoved. Full pytest (real builds cloning citusdata/docker) not run locally — it runs in CI.Diff stat: 3 files changed, 53 insertions(+), 48 deletions(-).