Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-17659

Improve Elasticsearch bulk indexing error handling

    XMLWordPrintable

    Details

      Description

      If ES is overloaded the bulk indexing command can be rejected and an error is logged, for instance:

      ERROR [Nuxeo-Work-elasticSearchIndexing-3] [org.nuxeo.elasticsearch.core.ElasticSearchIndexingImpl] failure in bulk execution: [0]: index [nuxeo], type [doc], id [########-####-####-############], message [RemoteTransportException[[????][inet[/#.###.###.###:9300]][bulk/shard]]; nested: EsRejectedExceptionException[rejected execution (queue capacity 50) on org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1@1746faca]; ]

      The bulk indexing command should be rescheduled.

      PS: Note that the fix for the above rejected log is to tune ES by increasing the ES_HEAP_SIZE to prevent GC overhead then increase the threadpool.bulk.queue_size.

        Attachments

          Issue Links

          depends on

          Improvement - An improvement or enhancement to an existing feature or task. NXP-18666 Increase the default number of documents per Elasticsearch bulk request

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: