Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-29319

Allow using arbitrary file keys in S3

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.10-HF31, 11.3, 2021.0
    • Component/s: S3
    • Release Notes Summary:
      Objects can be stored in S3 with an arbitrary file key.
    • Epic Link:
    • Tags:
    • Upgrade notes:
      Hide

       

      In order to be able to use arbitrary file keys generated either by the provider or by a trusted upload client the new S3BlobProvider should be used and the key strategy should be set to "managed" (default key strategy is "digest"): 

      nuxeo.core.binarymanager=org.nuxeo.ecm.blob.s3.S3BlobProvider
nuxeo.core.blobstore.keyStrategy=managed

       

       

      Show
        In order to be able to use arbitrary file keys generated either by the provider or by a trusted upload client the new S3BlobProvider should be used and the key strategy should be set to "managed" (default key strategy is "digest"):  nuxeo.core.binarymanager=org.nuxeo.ecm.blob.s3.S3BlobProvider nuxeo.core.blobstore.keyStrategy=managed    

      Description

      Today we rely on usage of the md5 digest of blobs as object keys in S3.

      This however means we have to compute this digest everytime as well as copying files to perform the required renames which takes time.

      As an optimization, when dealing with a trusted client, we could allow using an arbitrary file key provided by the client.

        Attachments

          Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. NXP-29795 Fix CSV export with S3 blobs

          • Blocker - Blocks development and/or testing work, production could not run.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-30682 Create a migration procedure to fix S3 blobs having original blob provider prefix in their key

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. NXP-29404 Fix S3 Direct Upload for big files

          • Blocker - Blocks development and/or testing work, production could not run.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-30632 Fix blobProviderId still present when moving blob from a S3 blob provider to another S3 blob provider

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-29181 Upload / “complete” call taking > 5 mins

          • Major - Major loss of function.
          • Resolved
          is required by

          Bug - A problem which impairs or prevents the functions of the product. NXP-29496 Fix S3 Direct Upload for little files (SSE-KMS)

          • Blocker - Blocks development and/or testing work, production could not run.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXDRIVE-2278 Fix S3 direct upload key computation

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-29478 Fix S3 direct upload for files larger than 5GB

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-29806 S3 Direct Upload managed mode using ETag as key

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          Is referenced in

          [Captain Hook] PR for 1.9_10.10: #31 PR for 1.9_10.10: #31

          [Captain Hook] PR for 10.10: #4251 PR for 10.10: #4251

          [Captain Hook] PR for master: #4236 PR for master: #4236

          [Captain Hook] PR for master: #4597 PR for master: #4597

          [Captain Hook] PR for master: #4752 PR for master: #4752

          [Captain Hook] PR for master: #4769 PR for master: #4769

            Activity

              People

              • Votes:
                2 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 1 week
                  1w