Limit Redis usage

Why Redis is an issue

Redis has been proven to be a component that can create some challenge when we use it in an HA deployment:

• Redis clustering is not easy
  • Sentinel is complex and not perfect
  • Redis non FOOS cluster is not an option for several customers
• Redis being in memory
  • we can always have a risk of overflow (i.e. when we use it for queuing)
  • evaluation potential data loss in case of fail-over is complex

To be fair, the problem is also related to the way we use Redis and not only about Redis itself.

WorkManager

The WorkManager needs to be very resilient so that:

• we can add/kill Nuxeo nodes without losing anything
• we can recover the jobs

For that the Redis based implementation is not compliant, so we are moving to a nuxeo-mqeues based implementation that will be able to leverage Kafka or Chronical Queues.

Key Value Storage

An other usage of Redis is the TransientStore:

• binaries are stored on shared FS
• structures and keys are stored inside Redis via Cachemanager

TransientStore has been updated to rely on a KeyValue abstraction (NXP-21871) leading the way to be able to use an other backend than Redis (i.e. MongoDB see NXP-23030).

PubSub