Uploaded image for project: 'Nuxeo Drive '
  1. Nuxeo Drive
  2. NXDRIVE-2332

Poll for the batchId on a regular basic to prevent its purgation from the transient store

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.4.2
    • Fix Version/s: 5.2.0
    • Component/s: Remote client
    • Release Notes Summary:
      Improved Large File Upload via S3
    • Release Notes Description:
      Hide

      When uploading a large file through S3, given the server configuration in-place, it may happen that the `batchId` on the Nuxeo side is deleted from the transient store while the binary is still being sent to S3.

      To improve the situation a sysadmin could set higher values for those properties:

      • `nuxeo.transientstore.ttl`
      • `nuxeo.transientstore.ttl2`
      • `nuxeo.s3storage.transient.ttl`
      • `nuxeo.s3storage.transient.ttl2`

      On the Nuxeo Drive side, the `batchId` will now be pinged on a regular basis (every 55 minutes) to update its TTL and prevent its deletion from the transient store.

      Show
      When uploading a large file through S3, given the server configuration in-place, it may happen that the `batchId` on the Nuxeo side is deleted from the transient store while the binary is still being sent to S3. To improve the situation a sysadmin could set higher values for those properties: `nuxeo.transientstore.ttl` `nuxeo.transientstore.ttl2` `nuxeo.s3storage.transient.ttl` `nuxeo.s3storage.transient.ttl2` On the Nuxeo Drive side, the `batchId` will now be pinged on a regular basis (every 55 minutes) to update its TTL and prevent its deletion from the transient store.
    • Epic Link:
    • Tags:
    • Sprint:
      nxDrive 11.2.20
    • Story Points:
      2

      Description

      Situation

      When asking for a batchId, it will have a TTL of 1 hour. If no call are made to that batchId within 1 hour, it will be removed. This is a critical issue when uploading large files via S3 for instance, where the batchId TTL is never updated because the app talks directly to S3 and not to Nuxeo while uploading chunks.

      Solution

      Poll for the batchId every hour (or 55 minutes exactly) to prevent its removal from the transient store.
      It is what is done on Web UI side: ping the batchId every hour.

        Attachments

          Issue Links

          Is referenced in

          [Captain Hook] PR for master: #2713 PR for master: #2713

          [Captain Hook] PR for master: #2716 PR for master: #2716

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 15 minutes
                  15m