New central namespace

In ODS 2, there was a central namespace cd, alongside 3 namespace dedicated to the provisioning app: prov-cd, prov-dev and prov-test. In ODS 3, there is only one namespace ods, which contains the whole OpenDevStack installation. When updating from ODS 2 to ODS 3, you will need to move the services running in cd, prov-cd and prov-test to ods. In detail:

  • SonarQube is moving from cd to ods

  • Nexus is moving from cd to ods

  • Jenkins (Master and Webhook Proxy) is moving from prov-cd to ods

  • Provisioning App is moving from prov-test to ods

The Provisioning App will make use of the Jenkins instance in ods to provision new ODS projects.

Apart from the instances, also all build configurations and image streams are moving from cd to ods, which means all users need to pull images from the new namespace.

With that in mind, we can start the update procedure!

Update your OpenDevStack repositories

Updating repositories means that new refs from repositories under github.com/opendevstack are pushed into the repositories in your BitBucket instance.

To do so, run the following and select the version you want to install (3.x).

make sync-repos

If your OpenDevStack installation is based on a custom branch (such as 3.acme), then you need to create a pull request on BitBucket from 3.x into that custom branch now.

Now that the repositories are updated, you also need to modify the images and the running instances in OpenShift.

Backup

Before proceeding, it is advisable to make a backup of the existing OpenShift configuration. This can be done easily with Tailor:

# Backup CD project
tailor export -n cd > backup_CD.yml

# Backup provision app namespaces
tailor export -n prov-cd > backup_PROV_CD.yml
tailor export -n prov-dev > backup_PROV_DEV.yml
tailor export -n prov-test > backup_PROV_TEST.yml

Note that the executing user needs to have permissions to access all resources in the cd namespaces for this to work properly.

Tailor

Next, update Tailor. 3.x requires Tailor 1.2.2.

Configuration

Run:

make prepare-config

In 2.x, the Jenkins master base image was pulled from the openshift namespace. The agent base image could have been pulled from that namespace as well, or directly from a registry. In 3.x, the master can also be pulled directly from a registry. The advantage of this is that you do not need to worry about the import policy set in the openshift namespace, which can have suprising results (e.g. your base image never updating even though a fix has been released by RedHat). The behaviour is defined by a new configuration parameter, JENKINS_MASTER_BASE_FROM_IMAGE.

Further, in 2.x you had to pick (via JENKINS_AGENT_BASE_IMAGE) whether you wanted to use Dockerfile.centos7 or Dockerfile.rhel7. However, both files contained the same instructions. Therefore, those were combined into a single Dockerfile and you only need to choose which base image to use via JENKINS_AGENT_BASE_FROM_IMAGE (like before).

After you have updated/added/removed all parameters with your desired values, commit and push the result.

OpenShift resources

Next you need to update all OpenShift resources (such as DC or BC). Review the diff produced by Tailor carefully before applying changes.

In ods-core, run:

make install-ods-project
make install-jenkins
make install-nexus
make install-doc-gen

In ods-quickstarters, run:

make install-jenkins-agent

Provisioning App changes

With OpenDevStack version 3.x the "production" instance of the Provisioning App is moving from the prov-test to the ods namespace. The following steps describe how to perform the change:

  1. Backup Data to a local directory
    oc cp prov-test/<ods-provisioning-pod-name>:/opt/provision/history/ ./history

  2. Backup Data to a local directory
    oc cp prov-test/<ods-provisioning-pod-name>:/opt/provision/history/ ./history

  3. Install the Provisioning App in the ods namespace
    make install-provisioning-app

  4. Copy backed up data to new instance
    oc cp ./history ods/<ods-provisioning-pod-name>:/opt/provision

  5. Depending on your use of the Provisioning App you can either:

    1. delete all prov-* namespaces if you are only interested in running a "production" instance.

    2. only delete the data under /opt/provision/history in the prov-test namespace if you want to keep your setup.

SonarQube changes

With OpenDevStack version 3.x SonarQube is moving from the cd namespace to the ods namespace. The following outlines the procedure for SonarQube:

  1. Create the SonarQube build resources in ods using make apply-sonarqube-build

  2. Start build of SonarQube image in ods using make start-sonarqube-build

  3. Scale down the SonarQube pod in cd

  4. Create a backup of the old data using make ODS_NAMESPACE=cd backup-sonarqube

  5. Create the SonarQube deploy resources in ods using make apply-sonarqube-deploy

  6. Scale down the SonarQube pod in ods

  7. Upload backup to new Postgres pod: oc -n ods cp sonarqube/sonarqube.sql <sonarqube-postgresql-pod>:/var/lib/pgsql/

  8. Start a bash in the Postgres pod: oc rsh -n ods pod/<sonarqube-postgresql-pod> bash and start a psql session.

  9. Drop the sonarqube database (DROP DATABASE sonarqube) and create a new, empty one (CREATE DATABASE sonarqube OWNER sonarqube). Then quit from psql.

  10. Import the backup with psql sonarqube < sonarqube.sql

  11. Scale up the SonarQube pod in ods.

  12. At this stage, the new SonarQube instance is fully functional, but the Elasticsearch index is not correct and therefore no projects show on the dashboard initially. To fix this, log into SonarQube with an admin user and go to "Administration > System". Then, start a bash in the SonarQube pod: oc rsh -n ods pod/<sonarqube-pod> bash and remove /opt/sonarqube/data/es6. Afterwards, restart the server from the UI (there’s a button on the "Administration > System" page). Booting will take some time (depending on the amount of data to process) as a full re-index is performed.

  13. Call make configure-sonarqube to verify the new installation is correctly configured.

  14. Now delete the old SonarQube route in cd.

  15. Create a new route in ods with the same host as the old route so that projects using the old URL don’t break.

Secure route removal

Checking for secure routes has been removed from the core, but is still available at https://github.com/BIX-Digital/ods-contrib.