Uploaded image for project: 'Nuxeo Drive '
  1. Nuxeo Drive
  2. NXDRIVE-1973

Do not sync non-folderish documents without known digest algorithm

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.3.0
    • Fix Version/s: 4.4.2
    • Component/s: Synchronizer
    • Release Notes Summary:
      Sync of non-folderish documents improved
    • Release Notes Description:
      Hide

      Some non-folderish documents can have zero digest. This is sometimes the case with special documents using the LiveConnect addon. It may also happen in very rare cases when a server contribution may be broken.
      In such scenario, Drive cannot synchronize the file as it has no control over the file integrity. Such documents are now ignored instead of generating errors.

      Show
      Some non-folderish documents can have zero digest. This is sometimes the case with special documents using the LiveConnect addon. It may also happen in very rare cases when a server contribution may be broken. In such scenario, Drive cannot synchronize the file as it has no control over the file integrity. Such documents are now ignored instead of generating errors.
    • Epic Link:
    • Tags:
    • Sprint:
      nxDrive 11.1.30, nxDrive 11.1.31
    • Story Points:
      3

      Description

      Issue

      The issue can be reprocude by trying to sync a Live Connect document, see Sentry Issue: NUXEO-DRIVE-2HQ

      TypeError: getattr(): attribute name must be string
  File "nxdrive\engine\processor.py", line 276, in _execute
  File "nxdrive\engine\processor.py", line 1246, in _synchronize_remotely_created
  File "nxdrive\engine\processor.py", line 1313, in _create_remotely
  File "nxdrive\engine\processor.py", line 1068, in _download_content
  File "nxdrive\client\remote_client.py", line 756, in stream_content
  File "nxdrive\client\remote_client.py", line 341, in download
  File "nxdrive\client\remote_client.py", line 385, in check_integrity_simple
  File "nxdrive\utils.py", line 1169, in compute_digest
  File "lib\site-packages\nuxeo\utils.py", line 60, in get_digest_hash

      Logs before the error:

      DEBUG    nuxeo.client       Response from 'https://intranet.nuxeo.com/nuxeo/site/automation/NuxeoDrive.GetFileSystemItem': '{"id":"defaultFileSystemItemFactory#default#DOC_UID","parentId":"PARENT_UID","name":"Customer Overview of Nuxeo Cloud","folder":false,"creator":"USERID","lastContributor":"USERID","creationDate":1547652368696,"lastModificationDate":1547652368696,"canRename":true,"canDelete":true,"lockInfo":null,"path":"PATH","userName":"USERID","downloadURL":"nxfile/default/DOC_UID/blobholder:0/Customer%20Overview%20of%20Nuxeo%20Cloud","digestAlgorithm":null,"digest":"\\"x2jdlkqYTB8kzPmV7jH2KPtlR68/MTU3MzIyNDI0OTk1Nw\\"","canUpdate":true}' with headers {'Date': 'Thu, 27 Feb 2020 09:44:00 GMT', 'Content-Type': 'application/json;charset=UTF-8', 'Content-Length': '934', 'Connection': 'keep-alive', 'Set-Cookie': '...', 'ETag': '"3ce57ad5af56ba0a2c412070bc4d67fa"', 'Content-Disposition': 'attachment; filename=file', 'Accept-Ranges': 'bytes'} and cookies <RequestsCookieJar[Cookie(...]>
DEBUG    nxdrive.client.remote_client Downloading file from 'https://intranet.nuxeo.com/nuxeo/nxfile/default/DOC_UID/blobholder:0/Customer%20Overview%20of%20Nuxeo%20Cloud' to PosixPath('/Users/USER/.nuxeo-drive/.tmp/ENGINE_UID/DOC_UID/Customer Overview of Nuxeo Cloud') with digest='"x2jdlkqYTB8kzPmV7jH2KPtlR68/MTU3MzIyNDI0OTk1Nw"'
DEBUG    nuxeo.client       Calling 'https://intranet.nuxeo.com/nuxeo/nxfile/default/DOC_UID/blobholder:0/Customer%20Overview%20of%20Nuxeo%20Cloud' with headers={...}, params=None and cookies=<RequestsCookieJar[Cookie(...)]>
DEBUG    nuxeo.client       Response from 'https://accounts.google.com/ServiceLogin?service=wise&passive=1209600&continue=https://docs.google.com/presentation/d/GOOGLE_DOC_UID/edit?usp%3Ddrivesdk&followup=https://docs.google.com/presentation/d/GOOGLE_DOC_UID/edit?usp%3Ddrivesdk&ltmpl=slides': '<no content>' with headers {...} and cookies <RequestsCookieJar[Cookie(...)]>

      Analysis

      This is a document with a blob that is a folder synced with Google Drive (Live Connect).

      • The digest algorithm is None.
      • The digest is not hexadecimal, here is an example: "x2jdlkqYTB8kzPmV7jH2KPtlR68/MTU3MzIyNDI0OTk1Nw".

      Fix

      Add a fonction that guess the event type: is_live_connect(self, event) -> bool.
      Call this method in the Remote Watcher and filter out those events.

      We also need to found a way to remove stalled transfer of those kind. Perhaps in sqlite.py we should bump the schema version to N + 1 and do the removal in the migration step. And associated doc pair should be filtered too.

        Attachments

          Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. NXDRIVE-2140 Handle documents with non-standard or empty digest

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXDRIVE-2236 [Direct Edit] Should work with non-standard or empty digests

          • Blocker - Blocks development and/or testing work, production could not run.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days
                  2d