Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-26911

Remove post commit listeners

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 10.10
    • Fix Version/s: 11.1, 2021.0
    • Component/s: Events / Works
    • Release Notes Summary:
      Post-commit listeners have been converted to asynchronous listeners.
    • Tags:
    • Backlog priority:
      900
    • Upgrade notes:
      Hide

      From 11.1, the post commit event listeners were made either asynchronous or synchronous. We strongly recommend to do the same thing with any custom event listener.

      Later on, we will deprecate the post commit event listener execution mechanism relying on PostCommitEventExecutor, see NXP-27986.
      For this purpose, a warning is logged when running a post commit event listener to inform that its execution will soon be deprecated and advising to update the listener contributions to make them asynchronous with <listener async=\"true\"...>.

      The warning can be disabled with the following logger in NUXEO_SERVER/lib/log4j2.xml:

      <Logger name="org.nuxeo.ecm.core.event.impl.PostCommitEventExecutor" level="warn">
  <RegexFilter regex="Running post commit event listeners.*(?s).*" onMatch="DENY" onMismatch="NEUTRAL" />
</Logger>
      Show
      From 11.1, the post commit event listeners were made either asynchronous or synchronous. We strongly recommend to do the same thing with any custom event listener. Later on, we will deprecate the post commit event listener execution mechanism relying on PostCommitEventExecutor , see NXP-27986 . For this purpose, a warning is logged when running a post commit event listener to inform that its execution will soon be deprecated and advising to update the listener contributions to make them asynchronous with <listener async=\"true\"...> . The warning can be disabled with the following logger in NUXEO_SERVER/lib/log4j2.xml: <Logger name="org.nuxeo.ecm.core.event.impl.PostCommitEventExecutor" level="warn"> <RegexFilter regex="Running post commit event listeners.*(?s).*" onMatch="DENY" onMismatch="NEUTRAL" /> </Logger>
    • Team:
      PLATFORM
    • Sprint:
      nxplatform 11.1.17
    • Story Points:
      3

      Description

      The idea of the postCommit listener is to run just after the transaction and wait for completion with a short timeout.
      The listener must implement PostCommitEventListener and marked as async=false (optionally postcommit=true), other combination results in and Async call instead of PostCommit.

      Instead of using the WM, the listener is executed in a dedicated thread pool, the transaction thread wait for completion (at least 300ms) before returning. This enables to do some processing after the transaction and before returning the HTTP response.

      To mitigate the randomness (sometimes 0.3s is not enough), the timeout is set to 1min when running tests.

      Examples:

      • EventCounterListener
      • Bulk mode see below

      Post-commit listeners are too complex and inefficient:

      • they use a dedicated large thread pool that can create up to 100 threads
      • there is an overhead to reconnect events and session
      • under peak, it participates to collapse the platform by using more threads, transactions and connections
      • the “pseudo realtime” behavior is random depending on the time the main thread wait for a reply, this creates complex bugs

      Nothing prevents to do the same “pseudo realtime” behavior using directly the afterCompletion, this is already done for the elasticsearch indexing

      For these reasons there should be no usage or the postCommit listeners at least on CRUD operation that need to scale,
      -> the EventCounterListener should be removed or turn into an async listener.
      -> it might be ok to keep the one in the Bulk mode which is only used with the importer (and it can be disabled)
      -> check for other known postCommit listeners

        Attachments

          Issue Links

          causes

          Clean up - NXDRIVE-2011 Remove post commit listeners warning

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          depends on

          Improvement - An improvement or enhancement to an existing feature or task. NXP-6859 Add more monitoring features to the Admin center

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. NXP-25314 Do a POC for event sourcing stream producer

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-12768 Warn the developer when listeners are incorrectly registered

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. NXP-27986 Deprecate post commit event listener execution

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Clean up - NXP-28012 Deprecate the Simon counters

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is required by

          Bug - A problem which impairs or prevents the functions of the product. NXP-28033 Fix FT Kafka ERROR In log taggedVersionListener documentRestored

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-28190 Possible ConcurrentModificationException at startup from BlobProviderDocumentsUpdateListener

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Clean up - NXDOC-1975 Document the warning about post commit event listener future deprecation

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days, 4 hours
                  2d 4h