Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-28316

Keep the platform fully available while performing a re-indexing with alias

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.10-HF55, 2021.12
    • Component/s: Elasticsearch
    • Release Notes Summary:
      It is now possible to perform a full reindexing of the repository without any interruption of service.
    • Tags:
    • Upgrade notes:
      Hide

      It is now possible to perform a full reindexing of the repository without any interruption of service.
      This will work in the following case:

      • Nuxeo is configured to manage elastic aliases: elasticsearch.manageAlias.enabled=true
      • Reindexing is done using the Bulk Service see NXP-26032 (the WorkManager legacy reindexing behavior triggered by the admin UI is unchanged)

      The repository is reindexed into a new index, all update operations are propagated to the existing and new indexes, when the repository indexing is completed aliases are updated, and the old index can be manually deleted.
      It is possible to scale up Nuxeo nodes during reindexing (and to scale down once the bulk scroller is completed).
      Note that you need to ensure you have enough elastic disk space first.

      Show
      It is now possible to perform a full reindexing of the repository without any interruption of service. This will work in the following case: Nuxeo is configured to manage elastic aliases: elasticsearch.manageAlias.enabled=true Reindexing is done using the Bulk Service see NXP-26032 (the WorkManager legacy reindexing behavior triggered by the admin UI is unchanged) The repository is reindexed into a new index, all update operations are propagated to the existing and new indexes, when the repository indexing is completed aliases are updated, and the old index can be manually deleted. It is possible to scale up Nuxeo nodes during reindexing (and to scale down once the bulk scroller is completed). Note that you need to ensure you have enough elastic disk space first.
    • Team:
      PLATFORM
    • Sprint:
      nxplatform #43, nxplatform #44, nxplatform #45, nxplatform #46, nxplatform #47, nxplatform #48, nxplatform #49
    • Story Points:
      2

      Description

      There is currently several methods to perform re-indexing, with more or less service interruption and resilience, but none of them allow to fully rebuild an index without any service interruption. By no service interruption I mean:

      1/ I start a full re-indexing (that may last several hours if I am on a huge repository)
      2/ during that time I create or update content
      3/ while re-indexing is happening, my Elasticsearch queries are redirected to the old index and take into account the new repository updates
      4/ as soon as my index is rebuilt, Elasticsearch queries use the new index that contains all the latest updates. 

      The period during which index is not up to date should be minimaliazed at most, if not inexistant.

        Attachments

          Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. NXP-30755 Fix ES manage alias feature in HF55 with MongoDB

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. NXP-30416 Starting a node while reindexing should not update the write alias index

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. NXDOC-2399 Document the new reindexing without service interruption

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          Is referenced in

          [Captain Hook] PR for 2021: #296 PR for 2021: #296

          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: