Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-25538

Support Avro Confluent encoding for stream

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.3
    • Component/s: Streams
    • Release Notes Description:
      Hide

      A new Nuxeo stream codec avroConfluent is supported.
      This will encode the stream using Confluent Avro message, this requires to have a Confluent Avro Schema Registry service running (part of the Confluent Platform).
      Because you can not change the codec of existing Nuxeo Streams, make sure Nuxeo starts with new streams:
      either remove your existing Kafka topics or use a different prefix either erase the Chronicle Queue directory nxserver/data/stream.
      Then update your nuxeo.conf:

      confluent.schema_registry.enabled=true
confluent.schema_registry.urls=http://schema-registry:8081
# use it where you want
nuxeo.stream.audit.log.codec=avroConfluent
      Show
      A new Nuxeo stream codec avroConfluent is supported. This will encode the stream using Confluent Avro message, this requires to have a Confluent Avro Schema Registry service running (part of the Confluent Platform). Because you can not change the codec of existing Nuxeo Streams, make sure Nuxeo starts with new streams: either remove your existing Kafka topics or use a different prefix either erase the Chronicle Queue directory nxserver/data/stream . Then update your nuxeo.conf : confluent.schema_registry.enabled= true confluent.schema_registry.urls=http: //schema-registry:8081 # use it where you want nuxeo.stream.audit.log.codec=avroConfluent
    • Tags:
    • Sprint:
      nxcore 10.3.4, nxcore 10.3.5
    • Story Points:
      3

      Description

      The Confluent platform 5 provides a Kafka 2.0 with other services, there is an Avro Schema registry.

      The Confluent Avro encoding is slightly different than official Avro Message.

      Supporting Confluent Avro message enables to use an existing Schema registry service accessible in REST outside of Nuxeo.
      It also offers a way to run KSQL on our streams.

        Attachments

          Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. NXP-25670 Cannot use a stream codec for StreamWorkManager

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. NXP-25600 Upgrade Kafka client from 1.1.0 to latest stable 2.0.0

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day
                  1d