Uploaded image for project: 'Nuxeo ECM Build/Test Environment'
  1. Nuxeo ECM Build/Test Environment
  2. NXBT-1816

ovh1 (itslave, matrix) is saturating on disk IO

    XMLWordPrintable

    Details

    • Type: Problem
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Continuous Integration

      Description

      The slave itslave used for the IT build is constantly saturating on disk IO.

      Because the slave has 32 CPUs and plenty of memory every process is waiting on disk IO, this creates a weird slave profile that is subject of lots of random failure in UT, 

      average of sar -pd 5 for few minutes:

Average:          DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util
Average:          sda     14.69    269.01   1999.64    154.45    122.36   9389.62     68.08    100.00

      some investigation/monitoring must be done to understand what is happening.

        Attachments

          Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. NXBT-1831 Use of a 3-nodes cluster for QA

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-22796 Fix random UT failure on nuxeo-mqueue TestPatternQueuingChronicle.killConsumers

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link Datadog Notebook used for investigation

          Web Link Drive is failing but LSI MegaRAID controller does not detect it

          Web Link Incident request

          Web Link Original request for Technical Advice

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 6 hours
                  6h