Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-29496

Fix S3 Direct Upload for little files (SSE-KMS)

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 10.10-HF30, 11.1
    • Fix Version/s: 10.10-HF31, 11.3, 2021.0
    • Component/s: S3
    • Release Notes Summary:
      S3 Direct upload works with SSE-KMS enabled.
    • Release Notes Description:
      Hide

      Introduces new configuration property to setup SSE/KMS on the transient bucket (for direct upload):

      nuxeo.s3storage.transient.crypt.kms.key=<sse-kms-key-id>
      
      Show
      Introduces new configuration property to setup SSE/KMS on the transient bucket (for direct upload): nuxeo.s3storage. transient .crypt.kms.key=<sse-kms-key-id>
    • Tags:
    • Backlog priority:
      1,000

      Description

      With SSE-KMS enabled on S3 buckets, an object's key does not match the ETag leading to this

      2020-08-04T17:05:29,782 WARN  [http-nio-0.0.0.0-8080-exec-3] [org.nuxeo.ecm.blob.s3.S3BlobStore] Invalid S3 object digest, expected=50da1cb107d84d72caa94e80bdeb8b9d actual=a4e7b0e875e88f786334262c001ac611
      2020-08-04T17:05:29,783 ERROR [http-nio-0.0.0.0-8080-exec-3] [org.nuxeo.ecm.core.blob.ManagedBlob] Failed to access file: 50da1cb107d84d72caa94e80bdeb8b9d 

      This happens with files less than 5 MEGAbytes in size.

      harlan@harlan:~$ ./nuxeo-server-10.10-tomcat/bin/nuxeoctl showconf | grep s3
      Include template: /home/harlan/nuxeo-server-10.10-tomcat/templates/s3binaries
      - amazon-s3-direct-upload (version: 1.1.3 - id: amazon-s3-direct-upload-1.1.3 - state: started)
      - amazon-s3-online-storage (version: 1.9.11 - id: amazon-s3-online-storage-1.9.11 - state: started)
      Package template: s3binaries
      nuxeo.core.binarymanager=org.nuxeo.ecm.blob.s3.S3BlobProvider
      nuxeo.s3storage.bucket=harlan
      nuxeo.s3storage.crypt.serverside=true
      nuxeo.s3storage.region=us-east-2
      nuxeo.s3storage.transient.bucket=harlan-transient
      nuxeo.s3storage.transient.crypt.serverside=true
      nuxeo.s3storage.transient.region=us-east-2
      nuxeo.s3storage.transient.roleArn=arn:aws:iam::00000000000:role/harlan
      nuxeo.s3storage.useDirectUpload=true
      nuxeo.templates=default,s3binaries
       

       

       

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 4 hours
                  4h

                    PagerDuty

                    Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.