-
Type: Bug
-
Status: Resolved
-
Priority: Critical
-
Resolution: Fixed
-
Affects Version/s: 10.10
-
Fix Version/s: 10.10-HF49, 11.x, 2021.6
-
Component/s: TransientStore
Under heavy load, this scenario can be easily reproduced:
- upload blobs using the Batch upload manager
- the upload stores the blobs to the transient storage with a call to FileBlob#moveTo
- at the same time, the transient store GC deletes old blobs with the parent folders in the hierarchy
- deleting the parent folders make the moveFile fail because the complete destination path cannot be resolved
- It raises an error like
2020-12-02T15:35:18,666 ERROR [http-nio-127.0.0.1-8080-exec-1684] [org.nuxeo.ecm.webengine.app.WebEngineExceptionMapper] org.nuxeo.ecm.core.api.NuxeoException: java.nio.file.NoSuchFileException: /nuxeo_data/nux_et_24610_tmp/nxblob-4784577703768182154.tmp -> /nuxeo_data/nuxeodata_transient_BatchManagerCache/data/d4/7c/d47c710e874524fdac9fb8551b9fef3f org.nuxeo.ecm.core.api.NuxeoException: java.nio.file.NoSuchFileException: /nuxeo_data/nux_et_24610_tmp/nxblob-4784577703768182154.tmp -> /nuxeo_data/nuxeodata_transient_BatchManagerCache/data/d4/7c/d47c710e874524fdac9fb8551b9fef3f at org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore.putBlobs(KeyValueBlobTransientStore.java:493) ~[nuxeo-core-cache-10.10-HF21.jar:?] at org.nuxeo.ecm.automation.server.jaxrs.batch.Batch.addFile(Batch.java:185) ~[nuxeo-automation-server-10.10-HF27.jar:?] at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.addBlob(BatchUploadObject.java:323) ~[nuxeo-rest-api-server-10.10-HF27.jar:?] at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.uploadNoTransaction(BatchUploadObject.java:252) ~[nuxeo-rest-api-server-10.10-HF27.jar:?] at org.nuxeo.ecm.restapi.server.jaxrs.BatchUploadObject.upload(BatchUploadObject.java:183) ~[nuxeo-rest-api-server-10.10-HF27.jar:?]
because "/nuxeo_data/nuxeodata_transient_BatchManagerCache/data/d4" does not exist any after the GC cleanup
Possible fixes:
- never deletes empty folders by the transient store GC
- catch the NoSuchFileException error and create the missing folders to have a valid path (<= preferred one not to accumulate empty folders)