Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-28801

Report Stream lag and latency from Nuxeo

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 11.1, 2021.0
    • Component/s: Monitoring, Streams
    • Epic Link:
    • Tags:
    • Team:
      PLATFORM
    • Sprint:
      nxplatform 11.1.32, nxplatform 11.1.33, nxplatform 11.1.34, nxplatform 11.1.35, nxplatform 11.1.36
    • Story Points:
      0

      Description

      Today we can monitor the stream processing lag and latency outside of Nuxeo using stream.sh.

      pros:

      • can be used even when Nuxeo is down, only Kafka access is required
      • don't impact Nuxeo perf

      cons:

      • more complex deployment, stream.sh being a standalone process, it needs to be run on multiple nodes to handle HA
      • adding reporter implementation (JMX or Prometheus) requires specific implementation

       

      In order to simplify the deployment, It will be better if these metrics are reported directly by Nuxeo, we will also gain the JMX and Prometheus reporting.

      Because these metrics are global to the cluster we need to use the single producer pattern (a single thread reporting metrics in the entire cluster)
      which is easy to do when using Kafka.

       

        Attachments

          Issue Links

          depends on

          Improvement - An improvement or enhancement to an existing feature or task. NXP-28879 Provide an Unified Stream/Log Manager

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. NXP-29358 Fix stream consumer latency

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-28877 Introduce Namespaces for Stream and Consumer Group names

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. NXP-28619 Run Nuxeo Platform unit tests against Kafka

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-26416 Nuxeo Stream should expose latency to Prometheus

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day
                  1d