Add database testing against PostgreSQL 9.6 and MongoDB 3.4

Given the need to upgrade the versions of the tested databases on master:

A. Escaffre: We met with Solen in order to define the supported version we want to target for the the LTS 2017 for the various repository backends.
We simply took the latest release of each. In parenthesis is the current version used on master (may be some inexact values)

• SQL SERVER 2016 (2008)
• Oracle Database 12c Release 2: 12.2.0.1 (patchset as of March 2017[7
• POSTGRES 9.6 ( 9.5)
• MYSQL SERVER 5.7.19 (5.5)
• MONGODB 3.4 (3.2)
• MARIADB 10.3 (10.?)
• MARKLOGIC 9.0

We will start with covering the requested versions of PostgreSQL and MongoDB with a new testing infrastructure called "Jenkins build with custom Docker image". This will allow the developers to contribute themselves to the tested matrix.

Here are 2 POC (commented in French).

Voici deux exemples inspirés des jobs /master/test-FT_nuxeo-remote-webdriver et /Private/System/deploy-nuxeo-nightly. Suivis de liens vers de la doc et des exemples concrets.
Le premier exemple démarre un slave qui démarre l'image custom dans laquelle on lance les tests.
L'image aura été mise à disposition sur dockerpriv.nuxeo.com par exemple (en accès public en interne, et via les identifiants LDAP en externe).
https://dockerpriv.nuxeo.com/v2/_catalog

-------------- Jenkinsfile

currentBuild.setDescription("Custom build on custom Docker image")
timeout(time: 1, unit: 'HOURS') {
    timestamps {
        node('SLAVE') {
            git 'git@github.com:nuxeo/my-repository.git'
            docker.image('dockerpriv.nuxeo.com:443/nuxeo/myCustomImg').withMaven(maven: 'maven-3') {
                sh "mvn verify -Pcustomdb,myCustomImg..."
            } 
            // withMaven will discover the generated Maven artifacts, JUnit Surefire & FailSafe reports and FindBugs reports            
    }
}

Le deuxième est un peu plus complexe; il démarre un slave qui lance un docker compose.

Le job /Private/System/deploy-nuxeo-nightly utilise de surcroît des Dockerfile pour customiser l'image Docker (initialisation database par ex) et la pusher.

De nombreuses améliorations et variantes sont possibles. Par exemple, on peut builder les images docker et les pusher avec les références du build.

-------------- Jenkinsfile

currentBuild.setDescription("PostgreSQL custom")
timeout(time: 1, unit: 'HOURS') {
    timestamps {
        node('SLAVE') {
            stage 'clone'
            git 'git@github.com:nuxeo/nuxeo-sample-project.git'

            stage 'tests'
            sh """#!/bin/bash -xe
                  docker-compose -f docker-compose.yml --project-name $JOB_NAME-$BUILD_NUMBER pull
                  docker-compose -f docker-compose.yml --project-name $JOB_NAME-$BUILD_NUMBER up
            """

            stage 'results'
            step([$class: 'ArtifactArchiver', artifacts: '**/target/failsafe-reports/*, **/target/*.png, **/target/**/*.log, **/target/**/log/*, fingerprint: false])
            step([$class: 'JUnitResultArchiver', testResults: '**/target/failsafe-reports/*.xml'])
        }
    }
}

-------------- docker-compose.yml

version: "2"
services:
  nuxeo:
    image: nuxeo:latest
    hostname: nuxeo
    links:
    - redis
    - es
    - db
    environment:
      # LAUNCHER_DEBUG: "-Djvmcheck=nofail"
      # NUXEO_PACKAGES: "..."
      # ...

  redis:
    image: redis:4.0.1

  es:
    image: elasticsearch:2.3.5

  db:
    image: dockerpriv.nuxeo.com:443/nuxeo/pgsql:9.5

  tests:
     image: dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave
     command: mvn -f /opt/workspace/pom.xml verify -Pcustomdb,myCustomImg... 
     volumes:
        - .:/opt/workspace/
    links:
    - redis
    - es
    - db
    environment:
        - NX_DB_HOST	db
        - NX_DB_PASS	nuxeo
        - NX_DB_USER    nuxeo
        - NX_PGSQL_DB_NAME	nuxeo
        - NX_PGSQL_DB_PORT	5432
        - NX_PGSQL_DB_ADMINNAME	template1
        - NX_PGSQL_DB_ADMINPASS	nuxeo
        - NX_PGSQL_DB_ADMINUSER	nxadmin

--------------- Alternatives

It is also possible to execute the commands from the Jenkins slave: that requires to retrieve networks values. 
For instance: 
    sh "DB=`docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' db`"

--------------- Improvements

Specify the network for interaction and usage from outside the docker compose.
Isolate from other builds with COMPOSE_PROJECT_NAME

Documentation et exemples.

Documentation:
    https://jenkins.io/doc/pipeline/steps/
    https://jenkins.io/doc/book/pipeline/docker/
    https://docs.docker.com/compose/reference/
    https://docs.docker.com/compose/compose-file/compose-file-v2/
    https://www.cloudbees.com/blog/top-10-best-practices-jenkins-pipeline-plugin
    https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/index.html#docker-workflow
    

Exemples:

    https://qa.nuxeo.org/jenkins/job/master/job/test-FT_nuxeo-remote-webdriver/
    
    https://github.com/nuxeo/nuxeo-tools-docker-priv
    https://qa.nuxeo.org/jenkins/job/Private/job/System/job/deploy-nuxeo-nightly/

    https://github.com/nemerosa/jenkins-docker/blob/master/Jenkinsfile

    https://github.com/jenkinsci/docker-workflow-plugin/blob/master/demo/repo/flow.groovy

    http://blog.arungupta.me/deployment-pipeline-docker-jenkins-java-couchbase/
    https://github.com/arun-gupta/docker-jenkins-pipeline/blob/master/Jenkinsfile