Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-30335

Evaluate Verctorized Redpanda as Kafka alternative

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Streams

      Description

      Redpanda is a C++ implementation of Kafka API that doesn't require Zookeeper.

      It could be an interesting alternative because it is easier to operate.

      Unit test on nuxeo-stream are failing with 17 errors.

      A first problem is that Redpanda is adding a control batch record in each partition every time there is a leader election (node started, topic created, network problem ...)

      Control batch records are special records that are not transmitted to the application, still, these records have offsets and this impacts the lag metric.
      Normally control batch records are used only with Kafka transaction which is not the case for Nuxeo.

      This is a serious problem for any system that relies in the lag metric to alert on consumer failures.
      https://github.com/vectorizedio/redpanda/issues/1135

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              bdelbosc Benoit Delbosc
              Participants:
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: