Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-30258

getting errors when creating files using aspera connector

    XMLWordPrintable

    Details

      Description

      we are currently testing a large upload to mimic a use case from a client where they want to use apsera to upload an external hard drive. We have gotten the files and copied the upload locally. it totallys 5k files comprised of folders and files. The files are xmp, cr2 and jpg.

      we can upload the cotnent to aspera without much incident, but when we try to complete the transfer (which creates the content in the nuxeo repo) we are seeing errors related to file not found.

      Stack trace is 

      021-03-11T18:00:05,151 ERROR [Nuxeo-Work-asperaCompletion-3:transfer:edf799e2-6fca-42b3-91aa-c0cc91ecfede:file:/edf799e2-6fca-42b3-91aa-c0cc91ecfede/Louise/Sp21_RN_React_Infinity_2_Louise_Matich_Photography_FK_0017.cr2:] [com.nuxeo.aspera.connector.service.AsperaCompleteWork] Error processing transfer: edf799e2-6fca-42b3-91aa-c0cc91ecfede, file: com.nuxeo.aspera.connector.api.FileInfo@20f5cf73[path=/edf799e2-6fca-42b3-91aa-c0cc91ecfede/Louise/Sp21_RN_React_Infinity_2_Louise_Matich_Photography_FK_0017.cr2,underFolder=true]
org.nuxeo.ecm.core.api.NuxeoException: java.nio.file.NoSuchFileException: /opt/nuxeo/server/tmp/nxbincache.3795004319554618696/bin_8572595242177520493.tmp
    at org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore.putBlobs(KeyValueBlobTransientStore.java:493) ~[nuxeo-core-cache-10.10-HF42.jar:?]
    at org.nuxeo.ecm.automation.server.jaxrs.batch.Batch.addFile(Batch.java:185) ~[nuxeo-automation-server-10.10-HF37.jar:?]
    at org.nuxeo.ecm.core.storage.sql.S3DirectBatchHandler.completeUpload(S3DirectBatchHandler.java:301) ~[nuxeo-core-binarymanager-s3-10.10-HF42.jar:?]
    at com.nuxeo.aspera.connector.service.AsperaCompleteWork.bmExecute(AsperaCompleteWork.java:192) ~[nuxeo-aspera-core-3.0.3-SNAPSHOT-PR-305.jar:?]
    at com.nuxeo.aspera.connector.service.AsperaCompleteWork.work(AsperaCompleteWork.java:82) [nuxeo-aspera-core-3.0.3-SNAPSHOT-PR-305.jar:?]
    at org.nuxeo.ecm.core.work.AbstractWork.runWorkWithTransaction(AbstractWork.java:493) [nuxeo-core-event-10.10-HF39.jar:?]
    at org.nuxeo.ecm.core.work.AbstractWork.run(AbstractWork.java:383) [nuxeo-core-event-10.10-HF39.jar:?]
    at org.nuxeo.ecm.core.work.WorkHolder.run(WorkHolder.java:57) [nuxeo-core-event-10.10-HF39.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: java.nio.file.NoSuchFileException: /opt/nuxeo/server/tmp/nxbincache.3795004319554618696/bin_8572595242177520493.tmp
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:1.8.0_242]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_242]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_242]
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) ~[?:1.8.0_242]
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) ~[?:1.8.0_242]
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) ~[?:1.8.0_242]
    at java.nio.file.Files.readAttributes(Files.java:1737) ~[?:1.8.0_242]
    at java.nio.file.Files.size(Files.java:2332) ~[?:1.8.0_242]
    at org.nuxeo.ecm.core.blob.LocalBlobStore.writeBlobGeneric(LocalBlobStore.java:63) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.AbstractBlobStore.writeBlob(AbstractBlobStore.java:124) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.CachingBlobStore.writeBlobGeneric(CachingBlobStore.java:130) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.AbstractBlobStore.writeBlob(AbstractBlobStore.java:124) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.AbstractBlobStore.writeBlob(AbstractBlobStore.java:115) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.BlobStoreBlobProvider.writeBlob(BlobStoreBlobProvider.java:117) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.BlobStoreBlobProvider.writeBlob(BlobStoreBlobProvider.java:127) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore.putBlobs(KeyValueBlobTransientStore.java:491) ~[nuxeo-core-cache-10.10-HF42.jar:?]
    ... 10 more
2021-03-11T18:00:06,117 ERROR [Nuxeo-Work-asperaCompletion-4:transfer:edf799e2-6fca-42b3-91aa-c0cc91ecfede:file:/edf799e2-6fca-42b3-91aa-c0cc91ecfede/Louise/Sp21_RN_React_Infinity_2_Louise_Matich_Photography_FK_2901.cr2:] [com.nuxeo.aspera.connector.service.AsperaCompleteWork] Error processing transfer: edf799e2-6fca-42b3-91aa-c0cc91ecfede, file: com.nuxeo.aspera.connector.api.FileInfo@2b8fe9a7[path=/edf799e2-6fca-42b3-91aa-c0cc91ecfede/Louise/Sp21_RN_React_Infinity_2_Louise_Matich_Photography_FK_2901.cr2,underFolder=true]
org.nuxeo.ecm.core.api.NuxeoException: java.nio.file.NoSuchFileException: /opt/nuxeo/server/tmp/nxbincache.3795004319554618696/bin_287998918757633687.tmp
    at org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore.putBlobs(KeyValueBlobTransientStore.java:493) ~[nuxeo-core-cache-10.10-HF42.jar:?]
    at org.nuxeo.ecm.automation.server.jaxrs.batch.Batch.addFile(Batch.java:185) ~[nuxeo-automation-server-10.10-HF37.jar:?]
    at org.nuxeo.ecm.core.storage.sql.S3DirectBatchHandler.completeUpload(S3DirectBatchHandler.java:301) ~[nuxeo-core-binarymanager-s3-10.10-HF42.jar:?]
    at com.nuxeo.aspera.connector.service.AsperaCompleteWork.bmExecute(AsperaCompleteWork.java:192) ~[nuxeo-aspera-core-3.0.3-SNAPSHOT-PR-305.jar:?]
    at com.nuxeo.aspera.connector.service.AsperaCompleteWork.work(AsperaCompleteWork.java:82) [nuxeo-aspera-core-3.0.3-SNAPSHOT-PR-305.jar:?]
    at org.nuxeo.ecm.core.work.AbstractWork.runWorkWithTransaction(AbstractWork.java:493) [nuxeo-core-event-10.10-HF39.jar:?]
    at org.nuxeo.ecm.core.work.AbstractWork.run(AbstractWork.java:383) [nuxeo-core-event-10.10-HF39.jar:?]
    at org.nuxeo.ecm.core.work.WorkHolder.run(WorkHolder.java:57) [nuxeo-core-event-10.10-HF39.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: java.nio.file.NoSuchFileException: /opt/nuxeo/server/tmp/nxbincache.3795004319554618696/bin_287998918757633687.tmp
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:1.8.0_242]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_242]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_242]
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) ~[?:1.8.0_242]
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) ~[?:1.8.0_242]
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) ~[?:1.8.0_242]
    at java.nio.file.Files.readAttributes(Files.java:1737) ~[?:1.8.0_242]
    at java.nio.file.Files.size(Files.java:2332) ~[?:1.8.0_242]
    at org.nuxeo.ecm.core.blob.LocalBlobStore.writeBlobGeneric(LocalBlobStore.java:63) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.AbstractBlobStore.writeBlob(AbstractBlobStore.java:124) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.CachingBlobStore.writeBlobGeneric(CachingBlobStore.java:130) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.AbstractBlobStore.writeBlob(AbstractBlobStore.java:124) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.AbstractBlobStore.writeBlob(AbstractBlobStore.java:115) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.BlobStoreBlobProvider.writeBlob(BlobStoreBlobProvider.java:117) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.blob.BlobStoreBlobProvider.writeBlob(BlobStoreBlobProvider.java:127) ~[nuxeo-core-api-10.10-HF42.jar:?]
    at org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore.putBlobs(KeyValueBlobTransientStore.java:491) ~[nuxeo-core-cache-10.10-HF42.jar:?]
    ... 10 more

        Attachments

          Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. NXP-30271 Key/Value Store GC shouldn't remove cache temporary files

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                lmcintyre Lisa McIntyre
                Participants:
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 3 hours
                  3h