Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-24640

Log appender sharding key should be used as Kafka record key

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 10.1
    • Fix Version/s: 9.10-HF04, 10.2
    • Component/s: Streams
    • Release Notes Description:
      Hide

      Using The Key as the Kafka Record Key

      When using the Kafka implementation of Nuxeo Stream, the key in the log is now the Kafka key itself, not the hash of it as it was previously (only option for the other backend, Chronicle queue).

      Show
      Using The Key as the Kafka Record Key When using the Kafka implementation of Nuxeo Stream, the key in the log is now the Kafka key itself, not the hash of it as it was previously (only option for the other backend, Chronicle queue).
    • Sprint:
      nxcore 10.2.1
    • Story Points:
      3

      Description

      Today org.nuxeo.lib.stream.log.LogAppender#append(java.lang.String, M) is appending a record to a Log using a hash of the provided key, the key itself is not saved.

      When using Kafka we could persist the key as the Kafka key record.

      This is not the case today because the notion of tuple (key, record) does not exists in Chronicle Queue, therefore if we need a key we encode it inside the record itself.

      The interest of having a Kafka key is when we want to play with log compaction or interact with other systems that rely on the Kafka tuple (key, record).

      This is also make sense for NXP-24512 so we can compact the latencies topic.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 4 hours
                  4h