Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-19631

Expose Elasticsearch Scroll API in ElasticSearchService

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 8.3
    • Component/s: Elasticsearch
    • Impact type:
      API change
    • Upgrade notes:
      Hide

      Added:

      • ElasticSearchService#scroll(NxQueryBuilder queryBuilder, long keepAlive)
      • ElasticSearchService#scanAndScroll(NxQueryBuilder queryBuilder, long keepAlive)
      • ElasticSearchService#scroll(EsScrollResult scrollResult)
      Show
      Added: ElasticSearchService#scroll(NxQueryBuilder queryBuilder, long keepAlive) ElasticSearchService#scanAndScroll(NxQueryBuilder queryBuilder, long keepAlive) ElasticSearchService#scroll(EsScrollResult scrollResult)
    • Sprint:
      nxfit 8.3.3

      Description

      The ElasticSearchService now exposes the Elasticsearch scroll API allowing to process large amounts of data.

      You can use it this way for instance:

              ElasticSearchService ess = Framework.getService(ElasticSearchService.class);
      
              // Perform initial search and get first batch of 20 results
              String query = "SELECT * FROM Document ORDER BY ecm:path";
              EsScrollResult scrollResult = ess.scroll(new NxQueryBuilder(session).nxql(query).limit(20), 10000);
      
              while (true) {
                  DocumentModelList batchOfDocs = scrollResult.getDocuments();
                  if (batchOfDocs.isEmpty()) {
                      // No more documents to handle
                      break;
                  }
                  for (DocumentModel doc : batchOfDocs) {
                      // Process document
                      ...
                  }
                  // Get next batch of results
                  scrollResult= ess.scroll(scrollResult);
              }
      

      Note: the keepAlive parameter in milliseconds only needs to be long enough to perform the next scroll query.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: