Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-27687

Add a DLQ fallback to computation policy

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Streams

      Description

      Today the computation policy can define a retry policy and a fallback which is Stop or Drop (continueOnFailure=true).
      It would be nice to be able to have an option to send the record in a Dead Letter Queue as well.
      Something like this:

          <streamProcessor name="myStreamProcessor" class="org.nuxeo.runtime.stream.tests.MyStreamProcessor" logConfig="default">
            <stream name="dlq-myComputation" partition="1" />
            <policy name="myComputation" maxRetries="10" delay="1s" maxDelay="60s" continueOnFailure="true" dlqStream="dlq-myComputation" />
      ...
       

      How to replay the records in the DQL is another story:

      • a simple way can be to copy the DLQ records into the original stream so they are reprocessed, this can be done using stream.sh dump and append.
      • have an option on the stream processor to start processing DLQ before input streams.
      • define a new recovery processor that reads from the DLQ (like it is done on the WorkManager repair procedure NXP-27148)

      Of course, for the 2 lasts options the DLQ mechanism should be disabled to not run in an infinite loop in case of systematic failure.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                bdelbosc Benoit Delbosc
                Participants:
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: