[NXP-29836] More robust blob upload to GCP - Nuxeo Issue Tracker

XML

Word

Printable

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 10.10
Fix Version/s: 10.10-HF37, 11.4, 2021.0
Component/s: BlobManager

Release Notes Summary:
Blobs are uploaded to Google Cloud Storage in chunks to benefit from the retriable API.
Release Notes Description:

Hide

We now upload Blobs to Google Cloud Storage in chunks. The default chunk size is 2MB and can be adjusted via storage.upload.chunk.size binary manager property or via nuxeo.gcp.storage.upload.chunk.size nuxeo.conf property.

Show
We now upload Blobs to Google Cloud Storage in chunks. The default chunk size is 2MB and can be adjusted via storage.upload.chunk.size binary manager property or via nuxeo.gcp.storage.upload.chunk.size nuxeo.conf property.
Tags:
- SupCom
- nxplatform
Backlog priority:
800
Team:
PLATFORM
Sprint:
nxplatform #22
Story Points:
3

Description

At least for the uploads, it would be safer to follow Google recommendation to use a retyr logic when an upload fails. This is described at https://cloud.google.com/storage/docs/exponential-backoff

Currently the upload to GCP storage could fail with this stacktrace

2020-11-05T01:41:38,061 ERROR [http-nio-127.0.0.1-8080-exec-82] [org.nuxeo.ecm.webengine.app.WebEngineExceptionMapper] com.google.cloud.storage.StorageException: 
com.google.cloud.storage.StorageException: 
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:227) ~[google-cloud-storage-1.76.0.jar:1.76.0]
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:308) ~[google-cloud-storage-1.76.0.jar:1.76.0]
	at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:178) ~[google-cloud-storage-1.76.0.jar:1.76.0]
	at com.google.cloud.storage.Bucket.create(Bucket.java:975) ~[google-cloud-storage-1.76.0.jar:1.76.0]
	at org.nuxeo.ecm.core.storage.gcp.GoogleStorageBinaryManager$GCPFileStorage.storeFile(GoogleStorageBinaryManager.java:185) ~[nuxeo-core-binarymanager-gcp-10.10-HF28.jar:?]
	at org.nuxeo.ecm.core.blob.binary.CachingBinaryManager.getBinary(CachingBinaryManager.java:155) ~[nuxeo-core-api-10.10-HF31.jar:?]
	at org.nuxeo.ecm.core.blob.binary.AbstractBinaryManager.getBinary(AbstractBinaryManager.java:130) ~[nuxeo-core-api-10.10-HF31.jar:?]
	at org.nuxeo.ecm.core.blob.binary.BinaryBlobProvider.writeBlob(BinaryBlobProvider.java:133) ~[nuxeo-core-api-10.10-HF31.jar:?]
	at org.nuxeo.ecm.blob.AbstractCloudBinaryManager.writeBlob(AbstractCloudBinaryManager.java:147) ~[nuxeo-core-binarymanager-common-10.10-HF21.jar:?]
	at org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore.putBlobs(KeyValueBlobTransientStore.java:491) ~[nuxeo-core-cache-10.10-HF21.jar:?]
	at org.nuxeo.ecm.automation.server.jaxrs.batch.Batch.addFile(Batch.java:185) ~[nuxeo-automation-server-10.10-HF29.jar:?]
	at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.addBlob(BatchUploadObject.java:332) ~[nuxeo-rest-api-server-10.10-HF30.jar:?]
	at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.uploadNoTransaction(BatchUploadObject.java:279) ~[nuxeo-rest-api-server-10.10-HF30.jar:?]
	at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.upload(BatchUploadObject.java:187) ~[nuxeo-rest-api-server-10.10

Could we add at least the retry logic in GCPFileStoragestoreFile ?