Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-30841

Improve indexing reliability when Elastic is overloaded

    XMLWordPrintable

    Details

    • Release Notes Summary:
      Nuxeo is now handling back pressure when Elastic is overloaded
    • Team:
      PLATFORM
    • Sprint:
      nxplatform #54
    • Story Points:
      3

      Description

      During document indexing with the WorkManager when Elastic is overloaded,
      we have different types of errors raised by the elastic:

      • indexing queue rejection
      • circuit breaker
      • 429 returned by AWS elastic

      As the result, the indexing work is either completed with failures or put in the DLQ and this creates a discrepancy between what is indexed and the repository content.

      Instead, a retry with a long delay should be performed to reduce the pressure.

      Another option could be to schedule a reindexing bulk command but this fallback processing may be too complex and it is going to be hard to evaluate if it works.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: