Allow to update all nodes of a Cluster

Goal

We want a way to define and deploy a set of Nuxeo Marketplace packages on all nodes of a cluster.

Typical use cases include :

• install an addon on all nodes
• install a hoftix on all nodes
• update a studio package on all nodes

This requirement exists :

• for nuxeo.io
• for all clients using a cluster

Configuration target and Orchestration

We have 2 tasks to achieve :

• define the target configuration each Node should apply
• trigger update on the Nuxeo nodes in the right order

Marketplace cluster channel

As explained in the nuxeo.io doc, the idea is to define a dedicated Marketplace channel :

• that is specific to a cluster group
  • ex : a studio project + HF + 1 JS app
• that constains update / new packages that *must be installed*
  • at startup NuxeoCtl must fetch and install any update

I think that using marketplace is a requirement :

• marketplace channel system is already pluggable
  • on the client and on the server side
• we should not create a new deployment model
  • install / upgrade / uninstall
• we are sure the cluster upgrade system is the same as the normal model
  • more tests and less bugs !

In addition, Marketplace package can be used for everything inside Nuxeo :

• bundles
• libs
• template and config files

The only direct limitation is that the system update are not handled, but I think that's a good point to focus on the Nuxeo application level only :

• this makes us independant from the underlying OS
• this makes the system usable for clients and not only for nuxeo.io

Orchestration

At the very low level, all we really need to do is : restart Nuxeo nodes in the right order.

We could probably do that using ansible or similar tool.

However, Mathieu started working with Redis for building a message bus between Cluster Nodes.

The good points of this approach are :

• we already have a Redis infrastructure for the Cluster
  • this is good to leverage this
• I like the idea of separating the system part from the application part
  • deploying ssh keys and ansible as part of Nuxeo Cluster infrastructure will be a pain for some clients
• Redis gives us a real communication bus with n-to-n
  • this will allow the nodes to push feedback

For now, what Mathieu started is here : https://github.com/nuxeo/nuxeo-cluster-controller

The target is to have :

• a central controller
  • talking to the agents
• small python agents that will be deployed on the Nuxeo nodes
  • talking to nuxeoctl and to the controller