Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-30471

Try to fix Kaniko Docker build failures

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 11.x, 2021.5
    • Component/s: CI/CD

      Description

      Randomly, the Kaniko Docker build fails for an unexplained reason:

      [2021-06-09T06:07:03.853Z] INFO[5805] cmd: EXPOSE                                  
[2021-06-09T06:07:03.853Z] INFO[5805] Adding exposed port: 8080/tcp                
[2021-06-09T06:07:03.853Z] INFO[5805] ENTRYPOINT ["/docker-entrypoint.sh"]         
[2021-06-09T06:07:03.853Z] INFO[5805] CMD ["nuxeoctl", "console"]                  
[2021-06-09T06:07:03.853Z] INFO[5805] USER 900                                     
[2021-06-09T06:07:03.853Z] INFO[5805] cmd: USER                                    
[2021-06-09T06:09:55.403Z] time="2021-06-09T06:09:51Z" level=fatal msg="build failed: build failed: building [nuxeo]: kaniko build for [nuxeo]: waiting for pod to complete: context closed while waiting for condition"
script returned exit code 1

      If often happens several times in a row.
      It could be due to a CPU/memory resource issue, when the K8s node pool is overloaded.

      See the work done in the AI builder to increase the Kaniko resources: https://github.com/nuxeo/jx-ai-builders/blob/master/skaffold.yaml#L26

      Or/and we could try these options:

      useNewRun
      snapshotmode

      Let's also add tolerations to be tolerated by the Platform dedicated node pool.

      Finally, we can try to upgrade to the latest version of Kaniko, it could also help for this issue, and maybe the COPY --chown issue, see NXP-28052.
       

        Attachments

          Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. NXBT-3523 Upgrade the skaffold version on jx-platform-builders

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-28052 Fix workaround for missing support of --chown flag with COPY command in Kaniko

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          Is referenced in

          [Captain Hook] PR for 2021: #182 PR for 2021: #182

          [Captain Hook] PR for master: #9 PR for master: #9

          [Captain Hook] PR for master: #4856 PR for master: #4856

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: