Uploaded image for project: 'Nuxeo Drive '
  1. Nuxeo Drive
  2. NXDRIVE-1973

Do not sync non-folderish documents without known digest algorithm

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.3.0
    • Fix Version/s: 4.4.2
    • Component/s: Synchronizer
    • Release Notes Summary:
      Sync of non-folderish documents improved
    • Release Notes Description:
      Hide

      Some non-folderish documents can have zero digest. This is sometimes the case with special documents using the LiveConnect addon. It may also happen in very rare cases when a server contribution may be broken.
      In such scenario, Drive cannot synchronize the file as it has no control over the file integrity. Such documents are now ignored instead of generating errors.

      Show
      Some non-folderish documents can have zero digest. This is sometimes the case with special documents using the LiveConnect addon. It may also happen in very rare cases when a server contribution may be broken. In such scenario, Drive cannot synchronize the file as it has no control over the file integrity. Such documents are now ignored instead of generating errors.
    • Epic Link:
    • Sprint:
      nxDrive 11.1.30, nxDrive 11.1.31
    • Story Points:
      3

      Description

      Issue

      The issue can be reprocude by trying to sync a Live Connect document, see Sentry Issue: NUXEO-DRIVE-2HQ

      TypeError: getattr(): attribute name must be string
        File "nxdrive\engine\processor.py", line 276, in _execute
        File "nxdrive\engine\processor.py", line 1246, in _synchronize_remotely_created
        File "nxdrive\engine\processor.py", line 1313, in _create_remotely
        File "nxdrive\engine\processor.py", line 1068, in _download_content
        File "nxdrive\client\remote_client.py", line 756, in stream_content
        File "nxdrive\client\remote_client.py", line 341, in download
        File "nxdrive\client\remote_client.py", line 385, in check_integrity_simple
        File "nxdrive\utils.py", line 1169, in compute_digest
        File "lib\site-packages\nuxeo\utils.py", line 60, in get_digest_hash
      

      Logs before the error:

      DEBUG    nuxeo.client       Response from 'https://intranet.nuxeo.com/nuxeo/site/automation/NuxeoDrive.GetFileSystemItem': '{"id":"defaultFileSystemItemFactory#default#DOC_UID","parentId":"PARENT_UID","name":"Customer Overview of Nuxeo Cloud","folder":false,"creator":"USERID","lastContributor":"USERID","creationDate":1547652368696,"lastModificationDate":1547652368696,"canRename":true,"canDelete":true,"lockInfo":null,"path":"PATH","userName":"USERID","downloadURL":"nxfile/default/DOC_UID/blobholder:0/Customer%20Overview%20of%20Nuxeo%20Cloud","digestAlgorithm":null,"digest":"\\"x2jdlkqYTB8kzPmV7jH2KPtlR68/MTU3MzIyNDI0OTk1Nw\\"","canUpdate":true}' with headers {'Date': 'Thu, 27 Feb 2020 09:44:00 GMT', 'Content-Type': 'application/json;charset=UTF-8', 'Content-Length': '934', 'Connection': 'keep-alive', 'Set-Cookie': '...', 'ETag': '"3ce57ad5af56ba0a2c412070bc4d67fa"', 'Content-Disposition': 'attachment; filename=file', 'Accept-Ranges': 'bytes'} and cookies <RequestsCookieJar[Cookie(...]>
      DEBUG    nxdrive.client.remote_client Downloading file from 'https://intranet.nuxeo.com/nuxeo/nxfile/default/DOC_UID/blobholder:0/Customer%20Overview%20of%20Nuxeo%20Cloud' to PosixPath('/Users/USER/.nuxeo-drive/.tmp/ENGINE_UID/DOC_UID/Customer Overview of Nuxeo Cloud') with digest='"x2jdlkqYTB8kzPmV7jH2KPtlR68/MTU3MzIyNDI0OTk1Nw"'
      DEBUG    nuxeo.client       Calling 'https://intranet.nuxeo.com/nuxeo/nxfile/default/DOC_UID/blobholder:0/Customer%20Overview%20of%20Nuxeo%20Cloud' with headers={...}, params=None and cookies=<RequestsCookieJar[Cookie(...)]>
      DEBUG    nuxeo.client       Response from 'https://accounts.google.com/ServiceLogin?service=wise&passive=1209600&continue=https://docs.google.com/presentation/d/GOOGLE_DOC_UID/edit?usp%3Ddrivesdk&followup=https://docs.google.com/presentation/d/GOOGLE_DOC_UID/edit?usp%3Ddrivesdk&ltmpl=slides': '<no content>' with headers {...} and cookies <RequestsCookieJar[Cookie(...)]>
      

      Analysis

      This is a document with a blob that is a folder synced with Google Drive (Live Connect).

      • The digest algorithm is None.
      • The digest is not hexadecimal, here is an example: "x2jdlkqYTB8kzPmV7jH2KPtlR68/MTU3MzIyNDI0OTk1Nw".

      Fix

      Add a fonction that guess the event type: is_live_connect(self, event) -> bool.
      Call this method in the Remote Watcher and filter out those events.

      We also need to found a way to remove stalled transfer of those kind. Perhaps in sqlite.py we should bump the schema version to N + 1 and do the removal in the migration step. And associated doc pair should be filtered too.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days
                  2d

                    PagerDuty

                    Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.