Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-29836

More robust blob upload to GCP

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 10.10
    • Fix Version/s: 10.10-HF37, 11.4, 2021.0
    • Component/s: BlobManager
    • Release Notes Summary:
      Blobs are uploaded to Google Cloud Storage in chunks to benefit from the retriable API.
    • Release Notes Description:
      Hide

      We now upload Blobs to Google Cloud Storage in chunks. The default chunk size is 2MB and can be adjusted via storage.upload.chunk.size binary manager property or via nuxeo.gcp.storage.upload.chunk.size nuxeo.conf property.

      Show
      We now upload Blobs to Google Cloud Storage in chunks. The default chunk size is 2MB and can be adjusted via storage.upload.chunk.size binary manager property or via nuxeo.gcp.storage.upload.chunk.size nuxeo.conf property.
    • Backlog priority:
      800
    • Team:
      PLATFORM
    • Sprint:
      nxplatform #22
    • Story Points:
      3

      Description

      At least for the uploads, it would be safer to follow Google recommendation to use a retyr logic when an upload fails. This is described at https://cloud.google.com/storage/docs/exponential-backoff

      Currently the upload to GCP storage could fail with this stacktrace

      2020-11-05T01:41:38,061 ERROR [http-nio-127.0.0.1-8080-exec-82] [org.nuxeo.ecm.webengine.app.WebEngineExceptionMapper] com.google.cloud.storage.StorageException: 
      com.google.cloud.storage.StorageException: 
      	at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:227) ~[google-cloud-storage-1.76.0.jar:1.76.0]
      	at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:308) ~[google-cloud-storage-1.76.0.jar:1.76.0]
      	at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:178) ~[google-cloud-storage-1.76.0.jar:1.76.0]
      	at com.google.cloud.storage.Bucket.create(Bucket.java:975) ~[google-cloud-storage-1.76.0.jar:1.76.0]
      	at org.nuxeo.ecm.core.storage.gcp.GoogleStorageBinaryManager$GCPFileStorage.storeFile(GoogleStorageBinaryManager.java:185) ~[nuxeo-core-binarymanager-gcp-10.10-HF28.jar:?]
      	at org.nuxeo.ecm.core.blob.binary.CachingBinaryManager.getBinary(CachingBinaryManager.java:155) ~[nuxeo-core-api-10.10-HF31.jar:?]
      	at org.nuxeo.ecm.core.blob.binary.AbstractBinaryManager.getBinary(AbstractBinaryManager.java:130) ~[nuxeo-core-api-10.10-HF31.jar:?]
      	at org.nuxeo.ecm.core.blob.binary.BinaryBlobProvider.writeBlob(BinaryBlobProvider.java:133) ~[nuxeo-core-api-10.10-HF31.jar:?]
      	at org.nuxeo.ecm.blob.AbstractCloudBinaryManager.writeBlob(AbstractCloudBinaryManager.java:147) ~[nuxeo-core-binarymanager-common-10.10-HF21.jar:?]
      	at org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore.putBlobs(KeyValueBlobTransientStore.java:491) ~[nuxeo-core-cache-10.10-HF21.jar:?]
      	at org.nuxeo.ecm.automation.server.jaxrs.batch.Batch.addFile(Batch.java:185) ~[nuxeo-automation-server-10.10-HF29.jar:?]
      	at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.addBlob(BatchUploadObject.java:332) ~[nuxeo-rest-api-server-10.10-HF30.jar:?]
      	at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.uploadNoTransaction(BatchUploadObject.java:279) ~[nuxeo-rest-api-server-10.10-HF30.jar:?]
      	at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.upload(BatchUploadObject.java:187) ~[nuxeo-rest-api-server-10.10 

      Could we add at least the retry logic in GCPFileStoragestoreFile ?

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day
                1d

                  PagerDuty

                  Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.