Uploaded image for project: 'Nuxeo Drive '
  1. Nuxeo Drive
  2. NXDRIVE-2183

Fix chunked upload using an obsolete batch ID in certain circumstances

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 4.0.0
    • Fix Version/s: 4.4.4
    • Component/s: Synchronizer
    • Release Notes Summary:
      Improved big file upload robustness
    • Release Notes Description:
      Hide

      We improved the robustness of big files upload. Dealing with files of 100 GB and more is always tricky due to all possible errors coming from the network stability, bandwith and the server availability. Many errors from the gateway timeout to the single chunk upload timeout have to be precisely and correctly handled.

      The situation is evolving with each and every releases, we are trying hard to make big files upload resilient and stable.

      Related tickets: NXPY-171, NXDRIVE-2185.

      Show
      We improved the robustness of big files upload. Dealing with files of 100 GB and more is always tricky due to all possible errors coming from the network stability, bandwith and the server availability. Many errors from the gateway timeout to the single chunk upload timeout have to be precisely and correctly handled. The situation is evolving with each and every releases, we are trying hard to make big files upload resilient and stable. Related tickets: NXPY-171 , NXDRIVE-2185 .
    • Epic Link:
    • Sprint:
      nxDrive 11.1.35
    • Story Points:
      1

      Description

      Error

      The scenario is quite complexe but when it happens (most likely with big files), it is quite hard to understand and debug.

      I managed to reproduce the issue by uploading a 100 GB file to the intranet.

      Step 1/3:

      • start the upload
      • it must fail at chunk N
      • the upload is then blacklisted and will be retried later

      Step 2/3:

      • for whatever reason, its batch ID is no more valid
      • resume the upload
      • a new batch ID is given
      • upload all chunks successfully
      • server error at linking the blob to the document
      • the upload is then blacklisted and will be retried later

      Step 3/3:

      • resume the upload
      • the batch ID used to resume the upload is the first batch ID, not the second that was used at step 2

      Fix

      The step 3 should simply work:

      • no chunks should be uploaded
      • the linking should work

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day
                  1d