Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-29427

Elasticsearch default configuration

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 10.10-HF29, 11.1
    • Fix Version/s: None
    • Component/s: Elasticsearch

      Description

      Fielddata and Nuxeo

      The current Nuxeo ES mapping uses fieldata on the dc:title field.

      => see default-doc-mapping.json

      This setting seems to be there for NXP-22808 and the idea is to be able to do partial (like) match on the title.

      However, as pointed in the ES doc, fielddata has an impact on memory usage at the ES level.

      Fielddata and ES Circuit-breaker

      During the benchmark, usage of fielddata triggered a Circuit-Breaker Exception because it required too much memory.

      see SUPINT-1744

       org.elasticsearch.ElasticsearchStatusException: 
        Elasticsearch exception [type=circuit_breaking_exception, 
         reason=[parent] Data too large, data for [<http_request>] would be [23679924882/22gb], which is larger than the limit of [22845023846/21.2gb], 
         usages [request=0/0b, fielddata=22159489269/20.6gb, in_flight_requests=2502/2.4kb, accounting=1520433111/1.4gb]]
      	at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)
      

      The solution to continue without increasing again the number of ES nodes was to re-index with a different mapping:

      See this mapping:https://github.com/nuxeo-sandbox/nuxeo-jit-blobstore/commit/59bbeb76555d5f64a672fd4425cccff9391a5cb2#diff-ebf15b720fb42a01d48d5ce4b23fdf5e

      Nuxeo default-configuration

      In its current state, the ES mapping does not allow us to scale, so we basically have 2 options:

      • we remove usage of fielddaete from the default mapping
      • we create a "large repository" dedicated mapping

      In both cases we need to:

      • document
      • define migration path

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  PagerDuty

                  Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.