Affects Version/s: None
The idea is to have a command to track periodically the latencies of consumers.
This latencies information can be replicated in a different datacenter and used to position the consumers in case of failover.
The latencies information are send into a Log, (for instance it can be named _consumer_latencies)
The latency is expressed
- with a key composed of:
- the consumer group
- log name
- and a value composed of:
- the date where the latency has been measured
- the timestamp of the last processed record
- eventually the key of the last processed record
This way another command can use this log to restore the consumer positions at a given time.
The rational for the composed key is to use the Kafka log compaction feature so the _consumer_latencies log can stay compact over time and the command to restore the position can quickly get the latest latency.