Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-32231

Create Management API to check indexing desync

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2023.7, 2021.49
    • Component/s: Elasticsearch
    • Release Notes Summary:
      There is a new management endpoint to check discrepancy between repository and elastic search
    • Sprint:
      nxplatform #104, nxplatform #105
    • Story Points:
      5

      Description

      We want to easily test an NXQL query against mongo and elastic in order to check a discrepancy and pinpoint an indexing problem.

      The proposed solution is to use 2 technical page providers one for the repo and the other for elastic, the page provider is taking an NXQL query and a pageSize.
      Both PP are queried and results are returned, because this is a management end point we are going to dump only the document ids (if needed use the page provider endpoint to get access to document content).

      Example of the usage:

      curl -s -u Administrator:Administrator --get \
         --data-urlencode 'nxql=SELECT * FROM Document WHERE ecm:isProxy = 0 AND ecm:isVersion = 0 AND ecm:isTrashed = 0' \
         --data-urlencode 'pageSize=5' \
        http://nuxeo.docker.localhost/nuxeo/api/v1/management/elasticsearch/checkSearch
      

      The json output:

      {
        "query": "SELECT * FROM Document WHERE ecm:isProxy = 0 AND ecm:isVersion = 0 AND ecm:isTrashed = 0",
        "order": {
          "sortColumn": "dc:modified",
          "sortAscending": false
        },
        "repo": {
          "took": 324,
          "resultsCount": 99891,
          "pageSize": 5,
          "pageProvider": "nxql_repo_search",
          "resultsCountLimit": 0,
          "result": [
            "545cb06b-3cef-428d-a9b8-a6069f2254c0",
            "373b3f32-ba4b-4ebd-b660-fd1385c78d59",
            "0755d7dc-e8b3-4610-805e-5d3bc60d8c77",
            "ac811414-7480-416f-a958-45b6a775a0f3",
            "7ec69eeb-0f28-4fed-befc-8598801e0b07"
          ]
        },
        "elastic": {
          "took": 22,
          "resultsCount": 99891,
          "pageSize": 5,
          "pageProvider": "nxql_elastic_search",
          "resultsCountLimit": 10000,
          "result": [
            "545cb06b-3cef-428d-a9b8-a6069f2254c0",
            "373b3f32-ba4b-4ebd-b660-fd1385c78d59",
            "0755d7dc-e8b3-4610-805e-5d3bc60d8c77",
            "7ec69eeb-0f28-4fed-befc-8598801e0b07",
            "ac811414-7480-416f-a958-45b6a775a0f3"
          ]
        }
      }
      

        Attachments

          Activity

            People

            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: