Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-17934

Improve audit performance and reliability

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 6.0
    • Fix Version/s: 8.3
    • Component/s: Audit
    • Tags:
    • Upgrade notes:
      Hide

      As the audit ingestion is bulked and is not based on asynch tasks, if you have to synchronize with the audit, you should use the new API org.nuxeo.ecm.platform.audit.api.AuditLogger.await(long, TimeUnit).

      In test mode, you could use the new API org.nuxeo.ecm.core.test.TransactionalFeature.nextTransaction(long, TimeUnit) which commit the current transaction and wait for asynch processing being terminated.

      Show
      As the audit ingestion is bulked and is not based on asynch tasks, if you have to synchronize with the audit, you should use the new API org.nuxeo.ecm.platform.audit.api.AuditLogger.await(long, TimeUnit) . In test mode, you could use the new API org.nuxeo.ecm.core.test.TransactionalFeature.nextTransaction(long, TimeUnit) which commit the current transaction and wait for asynch processing being terminated.
    • Sprint:
      nxSL Sprint 8.2.1, nxSL Sprint 8.3.1
    • Story Points:
      0

      Description

      On 7.4 audit log is saved in an async job using a pool with a single thread to serialize write access.

      On high throughput this process is a bit heavy and slow, for instance when using Redis:

      • there are 2 round trip to redis to save/get the job
      • there is a transaction and session open
      • the bundle events are reconnected
      • there is a bulk command send to ES
      • a commit

      In case of Redis failure the event is lost.
      In case of ES failure the event is lost (we can add a retry).

      This should be improved. A possible solution is to do like ES indexing: collect events in a sync listener then send them to ES during afterCompletion.
      In case of failure events can be logged to log4j and/or an async job can be scheduled to retry.

        Attachments

          Issue Links

          depends on

          Task - A task that needs to be done. NXBT-1018 Bench: Test with bigger default worker pool size

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is required by

          Bug - A problem which impairs or prevents the functions of the product. NXP-19523 Fix random failure on Selenium tests of "event log" tab

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. NXP-19472 process core events in audit inlined

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. NXBT-1028 Bench: Analyze audit and fulltext updater refactoring

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-17862 Improve fulltext extraction

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-19128 Reduce job serialization footprint

          • Major - Major loss of function.
          • Resolved

          Clean up - NXP-19160 remove completed jobs as soon as possible

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: