Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-26911

Remove post commit listeners

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 10.10
    • Fix Version/s: 11.1, 2021.0
    • Component/s: Events / Works
    • Release Notes Summary:
      Post-commit listeners have been converted to asynchronous listeners.
    • Backlog priority:
      900
    • Upgrade notes:
      Hide

      From 11.1, the post commit event listeners were made either asynchronous or synchronous. We strongly recommend to do the same thing with any custom event listener.

      Later on, we will deprecate the post commit event listener execution mechanism relying on PostCommitEventExecutor, see NXP-27986.
      For this purpose, a warning is logged when running a post commit event listener to inform that its execution will soon be deprecated and advising to update the listener contributions to make them asynchronous with <listener async=\"true\"...>.

      The warning can be disabled with the following logger in NUXEO_SERVER/lib/log4j2.xml:

      <Logger name="org.nuxeo.ecm.core.event.impl.PostCommitEventExecutor" level="warn">
        <RegexFilter regex="Running post commit event listeners.*" onMatch="DENY" onMismatch="NEUTRAL" />
      </Logger>
      
      Show
      From 11.1, the post commit event listeners were made either asynchronous or synchronous. We strongly recommend to do the same thing with any custom event listener. Later on, we will deprecate the post commit event listener execution mechanism relying on PostCommitEventExecutor , see NXP-27986 . For this purpose, a warning is logged when running a post commit event listener to inform that its execution will soon be deprecated and advising to update the listener contributions to make them asynchronous with <listener async=\"true\"...> . The warning can be disabled with the following logger in NUXEO_SERVER/lib/log4j2.xml: <Logger name="org.nuxeo.ecm.core.event.impl.PostCommitEventExecutor" level="warn"> <RegexFilter regex="Running post commit event listeners.*" onMatch="DENY" onMismatch="NEUTRAL" /> </Logger>
    • Team:
      PLATFORM
    • Sprint:
      nxplatform 11.1.17
    • Story Points:
      3

      Description

      The idea of the postCommit listener is to run just after the transaction and wait for completion with a short timeout.
      The listener must implement PostCommitEventListener and marked as async=false (optionally postcommit=true), other combination results in and Async call instead of PostCommit.

      Instead of using the WM, the listener is executed in a dedicated thread pool, the transaction thread wait for completion (at least 300ms) before returning. This enables to do some processing after the transaction and before returning the HTTP response.

      To mitigate the randomness (sometimes 0.3s is not enough), the timeout is set to 1min when running tests.

      Examples:

      • EventCounterListener
      • Bulk mode see below

      Post-commit listeners are too complex and inefficient:

      • they use a dedicated large thread pool that can create up to 100 threads
      • there is an overhead to reconnect events and session
      • under peak, it participates to collapse the platform by using more threads, transactions and connections
      • the “pseudo realtime” behavior is random depending on the time the main thread wait for a reply, this creates complex bugs

      Nothing prevents to do the same “pseudo realtime” behavior using directly the afterCompletion, this is already done for the elasticsearch indexing

      For these reasons there should be no usage or the postCommit listeners at least on CRUD operation that need to scale,
      -> the EventCounterListener should be removed or turn into an async listener.
      -> it might be ok to keep the one in the Bulk mode which is only used with the importer (and it can be disabled)
      -> check for other known postCommit listeners

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days, 4 hours
                  2d 4h

                    PagerDuty

                    Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.