Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-31273

Set a multi platform Docker Images build in CI

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2023.3, 2021.44
    • Component/s: Docker Image
    • Release Notes Summary:
      Docker Images are now built with Buildkit
    • Release Notes Description:
      Hide

      Nuxeo Docker images are now built with Buildkit instead of Kaniko.

      Buildkit allows us to handle the cross-platform aspect of Docker Images.

      Show
      Nuxeo Docker images are now built with Buildkit instead of Kaniko. Buildkit allows us to handle the cross-platform aspect of Docker Images.
    • Epic Link:
    • Team:
      PLATFORM
    • Sprint:
      nxplatform #96, nxplatform #97
    • Story Points:
      8

      Description

      We would like to build the Nuxeo Docker Images for the following platform: amd64 and arm64.

      This is needed for (at least) developers having an arm64 architecture. For instance on a MacBook Pro M1, the current Nuxeo Docker Image start in 150s instead of 30s.

      The proposed solution is to use the custom builder from Skaffold and leverage Docker BuildX to have this support.

      See below a script example:

      #!/bin/sh
      # This script demonstrates how to use `docker buildx` to build container
      # images for the linux/amd64 and linux/arm64 platforms.  It creates a
      # `docker buildx` builder instance when required.
      #
      # If you change the platforms, be sure to
      #
      #  (1) delete the buildx builder named `skaffold-builder`, and
      #  (2) update the corresponding node-affinities in k8s/pod.yaml.
      
      # The platforms to build. Default to linux/amd64.
      PLATFORMS=${PLATFORMS:=linux/amd64}
      
      # `buildx` uses named _builder_ instances configured for specific platforms.
      # This script creates a `skaffold-builder` as required.
      if ! docker buildx inspect skaffold-builder >/dev/null 2>&1; then
        docker buildx create --bootstrap\
            --name skaffold-builder\
            --driver=kubernetes\
            '--driver-opt="namespace=platform-staging","qemu.install=true","tolerations=key=team,value=platform"'\
            --platform $PLATFORMS
      fi
      
      # Building for multiple platforms requires pushing to a registry
      # as the Docker Daemon cannot load multi-platform images. 
      if [ "$PUSH_IMAGE" = true ]; then
        args="--platform $PLATFORMS --push"
      else
        args="--load"
      fi
      
      set -x      # show the command-line
      docker buildx build --builder skaffold-builder --tag $IMAGE $args "$BUILD_CONTEXT"
      

      And a Skaffold file update:

      apiVersion: skaffold/v2beta29
      kind: Config
      build:
        tagPolicy:
          envTemplate:
            template: "{{.VERSION}}"
        artifacts:
          - image: "$DOCKER_REGISTRY/nuxeo/builder-base-rockylinux9"
            platforms:
              - "linux/amd64"
              - "linux/arm64"
            context: rockylinux9
            custom:
              buildCommand: sh buildx.sh
      

      We will need to test the daemon aspect of Buildx, by default there're N Buildx pods that always run, is this would be sufficient for the build load, would creating several daemon (one for each major nuxeo version) is needed, would creating the Buildx pod before the build and clean it up after the build is needed.

      Note that this ticket is about changing the underlying skaffold implementation in order to support multi-platform build but not to leverage it, this is the scope of NXP-31815.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: