Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-21495

Specify event system design

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 9.1
    • Component/s: Events / Works, Streams

      Description

      Produce target architecture diagram

      The specification is to a use a computation pattern as described is Google Millwheel or Mesos Concord, this pattern is also used in Kafka Stream Processor API.

      The goal is to provide a pattern where it is easy to contribute computation (processor) and to compose them.
      The pattern does not require any distributed system knowledge, the code is viewed as a single threaded callback style mechanism which is easy to understand and debug.
      The computation rely on 2 callbacks one that is triggered on new record (message) and one triggered by a timer along with storage context
      this is enough to implement complex processing chain include batching (aggregation).

      This pattern can be implemented for standalone Nuxeo using nuxeo-mqueues relying on Chronicle queues.
      There will be some limitation compared to Concord that rely on Mesos to provide fault tolerance and delivery may be only at least one.

      By implementing a nuxeo-mqueues computation SPI using Kafka Processor API this will enable to work in Nuxeo cluster mode.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: