Today the computation policy can define a retry policy and a fallback which is Stop or Drop (continueOnFailure=true).
It would be nice to be able to have an option to send the record in a Dead Letter Queue as well.
Something like this:
How to replay the records in the DQL is another story:
- a simple way can be to copy the DLQ records into the original stream so they are reprocessed, this can be done using stream.sh dump and append.
- have an option on the stream processor to start processing DLQ before input streams.
- define a new recovery processor that reads from the DLQ (like it is done on the WorkManager repair procedure
Of course, for the 2 lasts options the DLQ mechanism should be disabled to not run in an infinite loop in case of systematic failure.