Uploaded image for project: 'Nuxeo Drive '
  1. Nuxeo Drive
  2. NXDRIVE-2598

Retry S3 non-chunked uploads on expired credentials

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 4.2.0
    • Fix Version/s: 5.1.1
    • Component/s: Remote client

      Description

      If that error happens, it will enter an infinite loop and the transfer will never finish:

      2021-04-09 14:34:46 5845 123145466855424 DEBUG    nxdrive.client.uploader Using <UploaderS3 is_complete=False, chunked=False, chunk_size=20971520, batch=<Batch batchId='batchId-42a5cf19-cedd-4b97-a6dc-2d24427e5d7c', blobs={}, dropped=None, etag=None, extraInfo={...}, key='3b2268ef-754e-4705-be5c-3a9d0fb102c0', multiPartUploadId=None, provider='s3', upload-idx=0>, blob=<FileBlob batchId='', chunkCount=0, fileIdx=None, mimetype='text/plain', name='2.txt', size=7, uploadType='normal', uploaded='true', uploadedChunkIds=[], uploadedSize=0>>
      2021-04-09 14:34:47 5845 123145466855424 WARNING  nxdrive.engine.processor Delaying failed upload
      Traceback (most recent call last):
        File "/.../nuxeo/handlers/s3.py", line 82, in upload
          response = self.s3_client.put_object(
        File "/.../botocore/client.py", line 357, in _api_call
          return self._make_api_call(operation_name, kwargs)
        File "/.../botocore/client.py", line 676, in _make_api_call
          raise error_class(parsed_response, operation_name)
      botocore.exceptions.ClientError: An error occurred (ExpiredToken) when calling the PutObject operation: The provided token has expired.
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "...engine/processor.py", line 342, in _execute
          self._handle_doc_pair_dt(doc_pair, sync_handler)
        File "...engine/processor.py", line 258, in _handle_doc_pair_dt
          sync_handler(doc_pair)
        File "...engine/processor.py", line 566, in _synchronize_direct_transfer
          self.remote.upload(
        File "...client/remote_client.py", line 493, in upload
          return uploader(self).upload(path, **kwargs)
        File "...client/uploader/direct_transfer.py", line 86, in upload
          item = super().upload_impl(
        File "...client/uploader/__init__.py", line 206, in upload_impl
          self.upload_chunks(transfer, blob, chunked)
        File "...client/uploader/__init__.py", line 313, in upload_chunks
          uploader.upload()
        File "/.../nuxeo/handlers/s3.py", line 89, in upload
          raise UploadError(self.blob.path, info=str(e))
      nuxeo.exceptions.UploadError: UploadError: unable to upload file '/Users/tiger-222/Downloads/a/fichiers/100/2.txt' (source: An error occurred (ExpiredToken) when calling the PutObject operation: The provided token has expired.)
      

      It only happens to non-chunked uploads through S3.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 hours
                  2h