Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-32582

Audit shouldn't index extended.aggregatesMatches

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Audit, Elasticsearch

      Description

      We encountered the following Elasticsearch error (client information masked): 

      Unable to index audit entry 0 : OpenSearchException[OpenSearch exception [type=mapper_parsing_exception, reason=failed to parse field [extended.aggregatesMatches.xxxxProductionNotes_datShot_agg.key] of type [date] in document with id '5659558'. Preview of field's value: '2024–03-11']]; nested: OpenSearchException[OpenSearch exception [type=illegal_argument_exception, reason=failed to parse date field [2024–03-11] with format [strict_date_optional_time||epoch_millis]]]; nested: OpenSearchException[OpenSearch exception [type=date_time_parse_exception, reason=date_time_parse_exception: Failed to parse with all enclosed parsers]];
      

      This happened when executing a search via a custom page provider. A 'search' entry is added to the audit log.The log entry includes the aggregation results of the page provider query including the field xxxxProductionNotes_datShot_agg.key.

      The property corresponding to this aggregate search field has 'string' data type. We also added a custom Elasticsearch mapping:

       

      "xxxProdNot:datShot": {
        "type": "keyword",
        "copy_to": "all_field",
        "ignore_above": 256,
        "fields": {
          "fulltext": {
            "analyzer": "fulltext",
            "type": "text"
          }
        }
      },
      

      However, Elasticsearch automatically derived data type 'date' for the xxxxProductionNotes_datShot_agg.key in the audit index, leading to above error when the string field does not parse as a date.

      Since we don't utilize the aggregation results from the audit log, would it be possible to change the audit logging and exclude aggregation results from the log entry?

      Another option could be changing the Elasticsearch mapping of the audit index, but this could be a more invasive operation. It may require creating a new audit index in Elasticsearch, copying old to new index. There is a risk of losing audit data in this process. In case we need to go with this option, can we get some guidance on the approach from the platform team?

      We are currently using LTS 2023 HF09.

       

       

       

       

       

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: