Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-30805

Route long indexing command to the Bulk Service keeping WM indexing near realtime

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.10-HF62, 2021.22
    • Component/s: Bulk
    • Backlog priority:
      800
    • Upgrade notes:
      Hide

      Use elasticsearch.index.recursive.bulkService=true in nuxeo.conf to activate the routing of long indexing command to the bulk service.
      In cluster mode, It is recommended to use the option when the bulk service is distributed (i.e. Kafka is enabled).

      Show
      Use elasticsearch.index.recursive.bulkService=true in nuxeo.conf to activate the routing of long indexing command to the bulk service. In cluster mode, It is recommended to use the option when the bulk service is distributed (i.e. Kafka is enabled).
    • Team:
      PLATFORM
    • Sprint:
      nxplatform #63
    • Story Points:
      3

      Description

      Outside of a full repository re-indexing case, there are actions like moving folder, changing ACL or deleting folder, that potentially trigger re-indexing of lots of documents.
      When this happens other indexing activities are delayed and this impacts directly GUI experience, for instance, a created document is not indexed and cannot be viewed.

      To avoid this we want to route all long indexing tasks into the bulk service, it acts as a separate indexing queue (distinct from the WorkManager indexing) avoiding delaying UI activity that requires near real time indexing.


      Initial case description to handle ACL change

      When an ACE is changed on a large folder (more than 500 documents) the Bulk service is used to materialize the Read ACL on the MongoDB side (since NXP-30426).
      The Elastic index also needs a materialization of the Read ACL, this is an orthogonal process that is currently relying on the WorkManager.
      Even if the Elastic update is reliable it uses the same indexing queue as the current activity and may delay too much its indexing.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  PagerDuty

                  Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.