-
Type: Improvement
-
Status: Resolved
-
Priority: Major
-
Resolution: Fixed
-
Affects Version/s: 10.10
-
Fix Version/s: 10.10-HF50, 11.x, 2021.7
-
Release Notes Summary:Enable concurrency on AuditLogWriter
-
Tags:
-
Backlog priority:800
-
Team:PLATFORM
-
Sprint:nxplatform #38, nxplatform #39, nxplatform #40
-
Story Points:1
Today the audit event listener (StreamAuditEventListener) in charge of writing the log entries to the audit stream is using the log entry id as a key to route the record to a partition.
But the log entry id at this stage is always 0 resulting in routing all the records to a single partition. This prevents using more than a single consumer in the cluster.
This is a problem only when the audit is specifically configured to use more than a single partition (default).
There are cases where we may want to use more than a single partition:
- consumers don't care about the ordering, this is the case for the audit log writer
- support higher throughput on audit log writer
Note that using more than one partition is not something that we want by default, this configuration provides a good default:
- preserve a total ordering of log entry events (even with multiple time desynchronized nodes)
- a good constant throughput without using more than 1 thread on the cluster
- prevent stressing the audit backend (elasticsearch) too much
Note that this is specific to 10.10 (LTS 2021 can use more than one partition) it also affects LTS 2021