Uploaded image for project: 'Nuxeo Documentation'
  1. Nuxeo Documentation
  2. NXDOC-1347

Defines upgrade steps for the ES 5.x migration and update all ES documentation

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 9.3
    • Component/s: Elasticsearch

      Description

      Defines the steps needed to upgrade an existing instance to Nuxeo 9.3 regarding the Elasticsearch 5.x support.
      Also update the Elasticsearch documentations page about mapping to follow ES 5 (at least 2 pages).

      So far the steps are:
      1. Upgrade Elasticsearch cluster from 2.x to 5.6.3
      2. Update custom contributions this includes settings, mapping to follow new ES 5 specification (see below)
      3. Reindex the repository so document are indexed with the new mapping
      4. Update custom code that query ES directly, check for passthrough usage or native ES queries (see below)

      TODO: Double check that audit and sequence indexes don't require any upgrade step

      Custom EP contribution must be adapted to use the new 3 EPs:

      • elasticSearchClient: To choose the type of client (RestClient, TransportClient) and how to access the ES cluster
      • elasticSearchIndex: To define the settings and mapping for an index (same as before)
      • elasticSearchEmbeddedServer: To start an embedded version of ES (only for dev purpose, not supported).

      Custom settings must be adapted to follow ES 5.x rules

      • settings can not contain anymore index directive (remove default shard and replication)
      • settings does not support anymore analyzer alias (removing the fulltext alias)

      Custom mapping must be adapted to follow ES 5.x rules

      • type multi_field must be rewritten to type keyword or text
      • type string must be rewritten to keyword (for not_analyzed) or text (the analyzed version)
      • not_analyzed is deprecated type should be keyword
      • text field using for sorting or aggregating must be explicitely define fielddata=true or be of keyword type
      • field used with NXQL LIKE requires a text field (will not work as default keyword), if the field is also used for sorting it must be fielddata ex for dc:title
            "dc:title": {
              "type": "text",
              "copy_to": "all_field",
              "fielddata": true,
              "fields": {
                "fulltext": {
                  "analyzer": "fulltext",
                  "type": "text",
                  "boost": 2
                }
              }
            },
        
      • _all is disabled. use a custom all_field instead, by default all string fields are copied to this field, to not index a field this requires to add a mapping without the "copy_to": "all_field" instruction.
      • highlight must be done on an analyzed fulltext field, so we need to use dc:title.fulltext and not dc:title

      Custom passthrough or native queries need to be reviewed to follow ES 5.x query format

      • searching with 1 or 0 on boolean field is deprecated, just this is only a warning at the moment, in 6.x it is going to be a problem
      • filtered query must be replaced with bool query
      • not filter should be replaced with a bool query
      • aggregate size=0 to get all buckets is not anymore supported the size must be explicetly defined (> 0)
      • There is no more default _all field. query_string and simple_query_string must use the new custom all_field field.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days, 35 minutes
                  2d 35m