Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-32485

Large folder copy should scale on DBS

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Core DBS
    • Tags:
    • Sprint:
      nxplatform #111
    • Story Points:
      5

      Description

      Today, the copy of folderish document is done synchronously at low level in DBSSession#copyRecurse this a recursive copy on all children level:
      https://github.com/nuxeo/nuxeo-lts/blob/2021/modules/core/nuxeo-core-storage-dbs/src/main/java/org/nuxeo/ecm/core/storage/dbs/DBSSession.java#L767C5-L777C6

      On large folder this can easily reach the default transaction timeout.

      This processing could be split into a sync processing that is limited and continued asynchronously.

      Because parent folder needs to be created before its children the copy must be ordered.

      A possible solution is to create the target folder tree synchronously, then run a bulk action to copy documents, this bulk action can be parallelized.

      Because this implementation it's not anymore synchronized it might create some regression and should be optional.

        Attachments

          Activity

            People

            • Assignee:
              bdelbosc Benoit Delbosc
              Reporter:
              bdelbosc Benoit Delbosc
              Participants:
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: