Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-29078

Stream AuditLogWriter processing cannot be parallelized



    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 10.10
    • Fix Version/s: 10.10-HF50, 11.x, 2021.7
    • Component/s: Audit, Streams
    • Release Notes Summary:
      Enable concurrency on AuditLogWriter
    • Backlog priority:
    • Team:
    • Sprint:
      nxplatform #38, nxplatform #39, nxplatform #40
    • Story Points:


      Today the audit event listener (StreamAuditEventListener) in charge of writing the log entries to the audit stream is using the log entry id as a key to route the record to a partition.

      But the log entry id at this stage is always 0 resulting in routing all the records to a single partition. This prevents using more than a single consumer in the cluster.

      This is a problem only when the audit is specifically configured to use more than a single partition (default).

      There are cases where we may want to use more than a single partition:

      • consumers don't care about the ordering, this is the case for the audit log writer
      • support higher throughput on audit log writer

      Note that using more than one partition is not something that we want by default, this configuration provides a good default:

      • preserve a total ordering of log entry events (even with multiple time desynchronized nodes)
      • a good constant throughput without using more than 1 thread on the cluster
      • prevent stressing the audit backend (elasticsearch) too much

      Note that this is specific to 10.10 (LTS 2021 can use more than one partition) it also affects LTS 2021




            • Votes:
              0 Vote for this issue
              6 Start watching this issue


              • Created: