Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-25424

Add a parameter to specify Kafka replication factor

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 10.1
    • Fix Version/s: 10.3
    • Component/s: Streams
    • Tags:
    • Sprint:
      nxcore 10.3.3
    • Story Points:
      1

      Description

      In order to specify the replication of Kafka topics, one has currently to override the default Kafka contribution like that:

      <?xml version="1.0"?>
      <component name="kafka.replicated.contrib">
      
        <require>org.nuxeo.kafka.defaultConfig</require>
      
        <extension point="kafkaConfig" target="org.nuxeo.runtime.stream.kafka.service">
          <kafkaConfig name="default" topicPrefix="${kafka.topicPrefix}">
            <producer>
              <property name="bootstrap.servers">${kafka.bootstrap.servers}</property>
              <property name="default.replication.factor">3</property>
            </producer>
            <consumer>
              <property name="bootstrap.servers">${kafka.bootstrap.servers}</property>
              <property name="request.timeout.ms">${kafka.request.timeout.ms}</property>
              <property name="max.poll.interval.ms">${kafka.max.poll.interval.ms}</property>
              <property name="session.timeout.ms">${kafka.session.timeout.ms}</property>
              <property name="heartbeat.interval.ms">${kafka.heartbeat.interval.ms}</property>
              <property name="max.poll.records">${kafka.max.poll.records}</property>
            </consumer>
          </kafkaConfig>
        </extension>
      </component>
      

      We should add a kafka.default.replication.factor parameter to make this configuration easier.

      As a side note, when no replication is set up, the LEADER_NOT_AVAILABLE error may be raised when a Kafka server goes down.

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 hour
                1h