Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-24641

All stream computations should use watermark

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 9.10, 10.1
    • Fix Version/s: 9.10-HF04, 10.2
    • Component/s: Streams
    • Release Notes Summary:
      All stream computations use watermark.
    • Release Notes Description:
      Hide

      Stream Computations Use Watermark

      Computations that re-append records in other streams now add a watermark (time and flag information) in the record. That allows always benefiting from advanced features of the streams such as latency computation.

      Show
      Stream Computations Use Watermark Computations that re-append records in other streams now add a watermark (time and flag information) in the record. That allows always benefiting from advanced features of the streams such as latency computation.
    • Epic Link:
    • Tags:
    • Sprint:
      nxcore 10.2.1
    • Story Points:
      3

      Description

      Computations work with special Record containing:

      • a string key
      • a watermark which a long that contains a timestamp + a sequence + a completed flag
      • flags
      • a payload which is a byte array.

      When appending a record to a stream using computation the the timestamp or watermark is optional.
      We should check that it is done for most of our usage otherwhise we cannot use advanced feature like latency tracking NXP-24512.

      The StreamWorkManager set a watermark equals to the submit time.
      But this is not the case for the AuditWriter:
      org.nuxeo.ecm.platform.audit.listener.StreamAuditEventListener#writeEntry should use the log entry timestamp or current time

        Attachments

          Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. NXP-24512 Extend the stream.sh to track consumer latencies

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 4 hours
                  4h