Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-30044

Async blob digest computation

    XMLWordPrintable

    Details

    • Release Notes Summary:
      A new property is added to compute blob digest asynchronously when Nuxeo doesn't see the blob content at upload time.
    • Release Notes Description:
      Hide

      The feature can be enabled with:

      nuxeo.core.blobstore.digestAsync=true
      Show
      The feature can be enabled with: nuxeo.core.blobstore.digestAsync= true
    • Epic Link:
    • Tags:
    • Team:
      FG
    • Sprint:
      nxFG 11.3.1
    • Story Points:
      5

      Description

      This issue deals with the actual background digest computation and blob replacement:

      • Log in a stream the keys of blobs that are copied (S3->S3) from a non-digest blob provider to one requiring digests.
      • Have an asynchronous computation for this stream to:
        • Download (streaming) the blob from S3 and compute its actual digest.
        • In the blob provider, copy the blob to a new one with the actual digest as key.
        • In the repository
          • find all blobs having the old key (using NXP-29516),
          • replace the old key with the new key,
          • optionally store the digest the high-level digest field.
        • In the blob provider, remove the old blob.

        Attachments

          Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. NXP-30618 Batch Upload fails when using gridfsbinaries template

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-30193 Fix cache access issue in CachingBlobStore

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-30231 Blob digest must be synthesized properly

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-30236 Update Aspera preview configuration with the new S3 Blob Provider

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. NXP-29806 S3 Direct Upload managed mode using ETag as key

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-29807 S3 Direct Upload using ETag as high-level blob digest

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. NXP-30276 Make customisable Aspera Blob configuration

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-32061 Fix incremental Blob GC when async digest is enabled

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-30158 Rely on the blob key for download using aspera

          • Major - Major loss of function.
          • Resolved
          is required by

          Bug - A problem which impairs or prevents the functions of the product. NXDRIVE-2140 Handle documents with non-standard or empty digest

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXDRIVE-2493 Enable app-to-local sync of ARW/PSB/PSD/CR2/NEF files on Web UI using S3

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-30271 Key/Value Store GC shouldn't remove cache temporary files

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-30328 Fix enabling of async digest on first start

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. NXP-30119 Process blobDigestUpdated event in Nuxeo Drive Server

          • Major - Major loss of function.
          • Resolved
          Is referenced in

          [Captain Hook] PR for 1.9_10.10: #32 PR for 1.9_10.10: #32

          [Captain Hook] PR for 10.10: #4615 PR for 10.10: #4615

          [Captain Hook] PR for master: #4752 PR for master: #4752

          [Captain Hook] PR for master: #4769 PR for master: #4769

          [Captain Hook] PR for master: #4924 PR for master: #4924

          [Captain Hook] PR for master: #4935 PR for master: #4935

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 weeks
                  2w