[NXP-32888] NuxeoDriveVirtualEventLogger should not write directly to the Audit backend - Nuxeo Issue Tracker

Details

Type: Bug
Status: Open
Priority: Minor
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: 2021.x, 2023.x, 2025.x
Component/s: Audit

Tags:
- nxplatform
- platform-review
Sprint:
nxplatform next
Story Points:
5

Description

The listener should not write directly to the audit backend, there should be only one writer in the cluster and that should be the audit/writer computation.

If another process is writing directly the audit then the audit/audit is no longer a reference source stream of audit events.

Also, audit backend write concurrency cannot be limited

Concurrency problem occurs on audit entry sequence both at the drive listener or audit/writer

work/commonPool-00,in:11524,inCheckpoint:11524,out:0,lastRead:1725540483984,lastTimer:0,wm:226170000456548353,loop:11603,checkpoint
org.nuxeo.ecm.core.api.ConcurrentUpdateException: [nuxeo-uidgen/Ui8A78ivSDmxIzD_F-4KlA][[nuxeo-uidgen][0]] OpenSearchStatusException[OpenSearch exception [type=version_conflict_engine_exception, reason=[audit]: version conflict, current version [307168323] is higher or equal to the one provided [307168322]]]
	at org.nuxeo.elasticsearch.client.ESRestClient.doIndex(ESRestClient.java:519) ~[nuxeo-elasticsearch-core-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.client.ESRestClient.lambda$index$11(ESRestClient.java:506) ~[nuxeo-elasticsearch-core-2023.15.7.jar:?]
	at net.jodah.failsafe.Functions.lambda$toSupplier$10(Functions.java:262) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.Execution.executeSync(Execution.java:128) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:379) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:212) ~[failsafe-2.4.4.jar:2.4.4]
	at org.nuxeo.elasticsearch.client.ESRestClient.index(ESRestClient.java:506) ~[nuxeo-elasticsearch-core-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.seqgen.ESUIDSequencer.initSequence(ESUIDSequencer.java:90) ~[nuxeo-elasticsearch-seqgen-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.seqgen.ESUIDSequencer.getNextBlockWithRetry(ESUIDSequencer.java:124) ~[nuxeo-elasticsearch-seqgen-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.seqgen.ESUIDSequencer.getNextBlock(ESUIDSequencer.java:112) ~[nuxeo-elasticsearch-seqgen-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.audit.ESAuditBackend.addLogEntries(ESAuditBackend.java:497) ~[nuxeo-elasticsearch-audit-2023.15.7.jar:?]
	at org.nuxeo.drive.listener.NuxeoDriveVirtualEventLogger.handleEvent(NuxeoDriveVirtualEventLogger.java:70) ~[nuxeo-drive-core-2023.15.7.jar:?]
	at org.nuxeo.ecm.core.event.impl.AsyncEventExecutor$ListenerWork.work(AsyncEventExecutor.java:209) ~[nuxeo-core-event-2023.15.7.jar:?]
	at org.nuxeo.ecm.core.work.AbstractWork.runWorkWithTransaction(AbstractWork.java:493) ~[nuxeo-core-event-2023.15.7.jar:?]
	at org.nuxeo.ecm.core.work.AbstractWork.run(AbstractWork.java:352) ~[nuxeo-core-event-2023.15.7.jar:?]
	Suppressed: org.nuxeo.ecm.core.api.ConcurrentUpdateException: [nuxeo-uidgen/Ui8A78ivSDmxIzD_F-4KlA][[nuxeo-uidgen][0]] OpenSearchStatusException[OpenSearch exception [type=version_conflict_engine_exception, reason=[audit]: version conflict, current version [307168374] is higher or equal to the one provided [307168372]]]
	at org.nuxeo.elasticsearch.client.ESRestClient.doIndex(ESRestClient.java:519) ~[nuxeo-elasticsearch-core-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.client.ESRestClient.lambda$index$11(ESRestClient.java:506) ~[nuxeo-elasticsearch-core-2023.15.7.jar:?]
	at net.jodah.failsafe.Functions.lambda$toSupplier$10(Functions.java:262) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.Execution.executeSync(Execution.java:128) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:379) ~[failsafe-2.4.4.jar:2.4.4]
	at net.jodah.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:212) ~[failsafe-2.4.4.jar:2.4.4]
	at org.nuxeo.elasticsearch.client.ESRestClient.index(ESRestClient.java:506) ~[nuxeo-elasticsearch-core-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.seqgen.ESUIDSequencer.initSequence(ESUIDSequencer.java:90) ~[nuxeo-elasticsearch-seqgen-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.seqgen.ESUIDSequencer.getNextBlockWithRetry(ESUIDSequencer.java:124) ~[nuxeo-elasticsearch-seqgen-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.seqgen.ESUIDSequencer.getNextBlock(ESUIDSequencer.java:112) ~[nuxeo-elasticsearch-seqgen-2023.15.7.jar:?]
	at org.nuxeo.elasticsearch.audit.ESAuditBackend.addLogEntries(ESAuditBackend.java:497) ~[nuxeo-elasticsearch-audit-2023.15.7.jar:?]
	at org.nuxeo.drive.listener.NuxeoDriveVirtualEventLogger.handleEvent(NuxeoDriveVirtualEventLogger.java:70) ~[nuxeo-drive-core-2023.15.7.jar:?]
	at org.nuxeo.ecm.core.event.impl.AsyncEventExecutor$ListenerWork.work(AsyncEventExecutor.java:209) ~[nuxeo-core-event-2023.15.7.jar:?]
	at

audit/writerPool-00,in:1260,inCheckpoint:1258,out:0,lastRead:1725539574495,lastTimer:1725539574119,wm:226169920110460929,loop:14314,record

Computation: audit/writer fails to process batch of 2 records after retries, last record: audit-audit-00:+7747906, policy: ComputationPolicy{maxRetries=20, delay=PT1S, delayMax=PT1M, continueOnFailure=false, batchCapacity=25, batchThreshold=PT0.5S}

org.opensearch.OpenSearchStatusException: OpenSearch exception [type=version_conflict_engine_exception, reason=[audit]: version conflict, current version [307165294] is higher or equal to the one provided [307165293]]
	at org.opensearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:202) ~[opensearch-1.3.17.jar:1.3.17]
	at org.opensearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:2075) ~[opensearch-rest-high-level-client-1.3.17.jar:1.3.17]
	at org.opensearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:2052) ~[opensearch-rest-high-level-client-1.3.17.jar:1.3.17]
...
org.nuxeo.elasticsearch.audit.ESAuditBackend.addLogEntries(ESAuditBackend.java:497) ~[nuxeo-elasticsearch-audit-2023.15.7.jar:?]
	at org.nuxeo.ecm.platform.audit.impl.StreamAuditWriter$AuditLogWriterComputation.writeEntriesToAudit(StreamAuditWriter.java:93) ~[nuxeo-platform-audit-core-2023.15.7.jar:?]
	at org.nuxeo.ecm.platform.audit.impl.StreamAuditWriter$AuditLogWriterComputation.batchProcess(StreamAuditWriter.java:79) ~[nuxeo-platform-audit-core-2023.15.7.jar:?]
	at org.nuxeo.lib.stream.computation.AbstractBatchComputation.batchProcess(AbstractBatchComputation.java:117) ~[nuxeo-stream-2023.15.7.jar:?]

NuxeoDriveVirtualEventLogger should not write directly to the Audit backend

Details

Description

Attachments

Activity

People

Dates