Component Pipeline

This pipeline allows to have a minimal Jenkinsfile in each repository by providing all language-agnostic build aspects. The goal is to duplicate as little as possible between repositories and have an easy way to ship updates to all projects.

Usage

Load the shared library in your Jenkinsfile like this:

@Library('ods-jenkins-shared-library@3.x') _

odsComponentPipeline(
  imageStreamTag: 'ods/jenkins-agent-golang:3.x',
  branchToEnvironmentMapping: [
    'master': 'dev',
    // 'release/': 'test'
  ]
) { context ->
  odsComponentFindOpenShiftImageOrElse(context) {
    stage('Build') {
      // custom stage
    }
    odsComponentStageScanWithSonar(context)
    odsComponentStageBuildOpenShiftImage(context)
  }
  odsComponentStageRolloutOpenShiftDeployment(context)
}

The version in @Library can be any Git revision, such as a branch (e.g. master or 2.x), a tag (e.g. v2.0) or even a specific commit.

There are many built-in stages like odsComponentStageScanWithSonar that you can use, please see Stages for more details.

Pipeline Options

odsComponentPipeline can be customized by passing configuration options like this:

odsComponentPipeline(
  imageStreamTag: 'ods/jenkins-agent-golang:3.x',
  dockerDir: 'foo'
)

Available options are:

Property Description

image

Container image to use for the Jenkins agent container. This value is not used when podContainers is set.

imageStreamTag

Container image tag of an ImageStream in your OpenShift cluster to use for the Jenkins agent container. This value is not used when podContainers or image is set.

alwaysPullImage

Determine whether to always pull the container image before each build run. Defaults to true. This value is not used when podContainers is set.

resourceRequestMemory

How much memory the container requests - defaults to 1Gi. This value is not used when podContainers is set.

resourceLimitMemory

Maximum memory the container can use - defaults to 2Gi. This value is not used when podContainers is set.

resourceRequestCpu

How much CPU the container requests - defaults to 10m. This value is not used when podContainers is set.

resourceLimitCpu

Maximum CPU the container can use - defaults to 300m. This value is not used when podContainers is set.

podLabel

Pod label, set by default to a random label to avoid caching issues. Set to a stable label if you want to reuse pods across builds.

podContainers

Custom pod containers to use if the default, automatically configured container is not suitable for your use case (e.g. if you need multiple containers such as app and database). See Agent customization.

podVolumes

Volumes to make available to the pod.

podServiceAccount

Serviceaccount to use when running the pod.

notifyNotGreen

Whether to send notifications if the build is not successful. Enabled by default.

emailextRecipients

Notify to this list of emails when notifyNotGreen is enabled. It is empty by default.

branchToEnvironmentMapping

Define which branches are deployed to which environments, see Git Workflow / Branch to Environment Mapping

projectId

Project ID, e.g. foo.

componentId

Component ID, e.g. be-auth.

environmentLimit

Number of environments to allow when auto-cloning environments.

dockerDir

The docker directory to use when building the image in openshift. Defaults to docker.

sonarQubeBranch

Please use option branch on odsComponentStageScanWithSonar.

failOnSnykScanVulnerabilities

Deprecated in 3.x! Please use option failOnVulnerabilities on odsComponentStageScanWithSnyk.

openshiftBuildTimeout

Deprecated in 3.x! Please use option buildTimeoutMinutes on odsComponentStageBuildOpenShiftImage.

openshiftRolloutTimeout

Deprecated in 3.x! Please use option deployTimeoutMinutes on odsComponentStageRolloutOpenShiftDeployment.

testResults

Configurable location for xunit test results, in case the build does not put them into build/test-results/test.

commitGitWorkingTree

Defaulting to false, if set to true, any changes in the working directory added with git add will be committed to the current branch and pushed.

Pipeline Context

When you write custom stages inside the closure passed to odsComponentPipeline, you have access to the context, which is assembled for you on the master node. The context can be influenced by changing the config map passed to odsComponentPipeline, see Pipeline Options.

The context object contains the following properties:

Property Description

jobName

Value of JOB_NAME. It is the name of the project of the build.

buildNumber

Value of BUILD_NUMBER. The current build number, such as 153.

buildUrl

Value of BUILD_URL. The URL where the results of the build can be found (e.g. http://buildserver/jenkins/job/MyJobName/123/)

buildTime

Time of the build, collected when the odsComponentPipeline starts.

credentialsId

Credentials identifier (Credentials are created and named automatically by the OpenShift Jenkins plugin).

tagversion

The tagversion is made up of the build number and the first 8 chars of the commit SHA.

nexusUrl

Nexus URL - value taken from NEXUS_URL. If NEXUS_URL is not present, it will default to NEXUS_HOST (which also includes the scheme). nexusHost is an alias for nexusUrl.

nexusUsername

Nexus username.

nexusPassword

Nexus password.

nexusUrlWithBasicAuth

Nexus URL, including username and password as BasicAuth. nexusHostWithBasicAuth is an alias for nexusUrlWithBasicAuth.

sonarQubeEdition

Edition of SonarQube in use, determined by SONAR_EDITION (defaults to community).

environment

The environment which was chosen as the deployment target, e.g. dev.

targetProject

Target project, based on the environment. E.g. foo-dev.

cdProject

CD project. E.g. foo-cd.

groupId

Group ID, defaults to: org.opendevstack..

projectId

Project ID, e.g. foo.

componentId

Component ID, e.g. be-auth.

selector

Selector common to all resources of component, defaults to app=${projectId}-${componentID} (e.g. app=foo-be-auth).

gitUrl

Git URL of repository

gitBranch

Git branch for which the build runs.

gitCommit

Git commit SHA to build.

shortGitCommit

Short Git commit SHA (first 8 chars) to build.

gitCommitAuthor

Git commit author.

gitCommitMessage

Git commit message (sanitized).

gitCommitRawMessage

Git commit message (raw).

gitCommitTime

Git commit time in RFC 3399.

issueId

Jira issue ID if any present in either commit message or branch name (e.g. 123 from commit message FOO-123: Bar or branch feature/FOO-123-bar). If the issue ID is present in both, the branch name has precedence.

openshiftHost

OpenShift host - value taken from OPENSHIFT_API_URL.

odsSharedLibVersion

ODS Jenkins shared library version, taken from reference in Jenkinsfile.

bitbucketUrl

Bitbucket URL - value taken from BITBUCKET_URL. If BITBUCKET_URL is not present, it will default to https://<BITBUCKET_HOST>`. bitbucketHost is an alias for bitbucketUrl.

dockerDir

The docker directory to use when building the image in openshift. Defaults to docker.

Git Workflow / Branch to Environment Mapping

The shared library does not impose which Git workflow you use. Whether you use git-flow, GitHub flow or a custom workflow, it is possible to configure the pipeline according to your needs by configuring the pipeline option branchToEnvironmentMapping. The setting could look like this:

branchToEnvironmentMapping: [
  'master': 'prod',
  'develop': 'dev',
  'hotfix/': 'hotfix',
  '*': 'review'
]

There are three ways to reference branches:

  • Fixed name (e.g. master)

  • Prefix (ending with a slash, e.g. hotfix/)

  • Any branch (*)

Matches are made top-to-bottom. For prefixes / any branch, a more specific environment might be selected if:

  • the branch contains a ticket ID and a corresponding env exists in OpenShift. E.g. for mapping "feature/": "dev" and branch feature/foo-123-bar, the env dev-123 is selected instead of dev if it exists.

  • the branch name corresponds to an existing env in OpenShift. E.g. for mapping "release/": "rel" and branch release/1.0.0, the env rel-1.0.0 is selected instead of rel if it exists.

Examples

If you use git-flow, the following config fits well:

branchToEnvironmentMapping: [
  'master': 'prod',
  'develop': 'dev',
  'release/': 'rel',
  'hotfix/': 'hotfix',
  '*': 'preview'
]

If you use GitHub Flow, the following config fits well:

branchToEnvironmentMapping: [
  'master': 'prod',
  '*': 'preview'
]

If you use a custom workflow, the config could look like this:

branchToEnvironmentMapping: [
  'production': 'prod',
  'master': 'dev',
  'staging': 'uat'
]

Advanced

Agent customization

The agent used in the pipeline can be customized by adjusting the image (or imageStreamTag to use. Further, alwaysPullImage (defaulting to true) can be used to determine whether this image should be refreshed on each build.

Resource constraints of the container can be changed via resourceRequestCpu, resourceLimitCpu, resourceRequestMemory and resourceLimitMemory.

The setting podVolumes allows to mount persistent volume claims to the pod (the value is passed to the podTemplate call as volumes).

To completely control the container(s) within the pod, set podContainers (which is passed to the podTemplate call as containers).

Configuring of a customized agent container in a Jenkinsfile:

odsComponentPipeline(
  branchToEnvironmentMapping: [:],
  podContainers: [
    containerTemplate(
      name: 'jnlp', // do not change, see https://github.com/jenkinsci/kubernetes-plugin#constraints
      image: "${env.DOCKER_REGISTRY}/foo-cd/jenkins-agent-custom",
      workingDir: '/tmp',
      resourceRequestCpu: '100m',
      resourceLimitCpu: '500m',
      resourceRequestMemory: '2Gi',
      resourceLimitMemory: '4Gi',
      alwaysPullImage: true,
      args: '${computer.jnlpmac} ${computer.name}'
    )
  ],
  ...
  ) { context ->
  stageBuild(context)
  ...
}

See the kubernetes-plugin documentation for possible configuration.

Git LFS (Git Large File Storage extension)

If you are working with large files (e.g.: binary files, media files, files bigger than 5MB…​), you can follow the following steps:

  • Check this HOWTO about Git LFS

  • Track your large files in your local clone, as explained in previous step

  • Enable Git LFS in your repository (if Bitbucket: under repository’s settings main page you can enable it)

NOTE: if already having a repository with large files and you want to migrate it to using git LFS:

git lfs migrate

Deploying OpenShift resources from source code

By default, the component pipeline uses existing OpenShift resources, and just creates new images / deployments related to them. However, it is possible to control all OpenShift resources in code, following the infrastructure-as-code approach. This can be done by defining the resources as OpenShift templates in the directory openshift of the repository, which will then get applied by Tailor when running the pipeline. The advantage of this approach:

  • All changes to OpenShift resources are traceble: who did the change and when?

  • Moving your application between OpenShift projects or even clusters is trivial

  • Changes to your application code that require a change in configuration (e.g. a new environment variable) as well can be done together in one commit.

If you have an existing component for which you want to enable this feature, you simply need to run:

mkdir -p openshift
tailor -n foo-dev export -l app=foo-bar > openshift/template.yml

Commit the result and the component pipeline should show in the ouput whether there has been drift and how it was reconciled.

When using this approach, you need to keep a few things in mind:

  • Any changes done in the OpenShift web console will effectively be reverted with each deploy. When you store templates in code, all changes must be applied to them.

  • You can always preview the changes that will happen by running tailor diff from your local machine.

  • DeploymentConfig resources allow to specify config and image triggers (and ODS configures them by default like this). When deploying via Tailor, it is recommended to remove the image trigger, otherwise you might trigger two deployments: one when config (such as an environment variable) changes, and one when the image changes. When you remove the image trigger, it is crucial to add the internal registry to the image field, and to configure imagePullPolicy: Always for the container (otherwise you might roll out old images).

If you want to use encrypted secrets with Tailor, you have to create a keypair for Jenkins so that the pipeline can use it to decrypt the parameters. The easiest way to do this is to create an OpenShift secret named tailor-private-key and sync it with Jenkins as a credential. Example:

tailor secrets generate-key jenkins@example.com
oc -n foo-cd create secret generic tailor-private-key --from-file=ssh-privatekey=private.key
oc -n foo-cd label secret tailor-private-key credential.sync.jenkins.openshift.io=true

Controlling your OpenShift resources in source code enables a lot of other use cases as well. For example, you might want to preview changes to a component before merging the source code. By using Tailor to deploy your templates, you can create multiple running components from one repository, e.g. one per feature branch. Following are some steps how to achieve this:

First, add 'feature/': 'dev' to the branchToEnvironmentMapping. Then, create new variables in the pipeline block:

def componentSuffix = context.issueId ? "-${context.issueId}" : ''
def suffixedComponent = context.componentId + componentSuffix

With this in place, you can adapt the rollout stage:

odsComponentStageRolloutOpenShiftDeployment(
  context,
  [
    tailorSelector: "app=${context.projectId}-${suffixedComponent}",
    tailorParams: ["COMPONENT_SUFFIX=${componentSuffix}"]
  ]
)

And finally, in your openshift/template.yml, you need to add the COMPONENT_SUFFIX parameter and append ${COMPONENT_SUFFIX} everywhere the component ID is used in deployment relevant resources (such as Service, DeploymentConfig, Route). That’s all you need to have automatic previews!

You might want to clean up when the code is merged, which can be achieved with something like this:

stage('Cleanup preview resources') {
  if (context.environment != 'dev') {
    echo "Not performing cleanup outside dev environment"; return
  }
  def mergedIssueId = org.ods.services.GitService.mergedIssueId(context.projectId, context.repoName, context.gitCommitRawMessage)
  if (mergedIssueId) {
    echo "Perform cleanup of suffix '-${mergedIssueId}'"
    sh("oc -n ${context.targetProject} delete all -l app=${context.projectId}-${context.componentId}-${mergedIssueId}")
  } else {
    echo "Nothing to cleanup"
  }
}

Interacting with Bitbucket

The shared library already sets the build status of the built commit. It also provides convenience methods on BitbucketService to interact with pull requests:

  • String createPullRequest(String repo, String fromRef, String toRef, String title, String description, List<String> reviewers) creates a pull request in repo from branch fromRef to toRef. reviewers is a list of bitbucket user names.

  • List<String> getDefaultReviewers(String repo) returns a list of bitbucket user names (not display names) that are listed as the default reviewers of the given repo.

  • String getDefaultReviewerConditions(String repo) returns all default reviewer conditions of the given repo, which can be parsed using readJSON.

  • String getPullRequests(String repo, String state = 'OPEN') returns all open pull requests, which can be parsed using readJSON.

  • Map findPullRequest(String repo, String branch, String state = 'OPEN') tries to find a pull request for the given branch, and returns a map with its ID and target branch.

  • void postComment(String repo, int pullRequestId, String comment) allows to add comment to the PR identified by pullRequestId.

To make use of these methods, you need to get an instance of the BitbucketService in your Jenkinsfile like this:

Jenkinsfile
import org.ods.services.ServiceRegistry
import org.ods.services.BitbucketService

def sayHello(def context) {
  stage('Say Hello') {
    def bitbucketService = ServiceRegistry.instance.get(BitbucketService)
    bitbucketService.postComment(context.repoName, 1, "Hello world")
  }
}

Skipping pipeline runs

If the subject of the built commit message contains [ci skip], [skip ci] or ***NO_CI***, the pipeline is skipped.

# skip pipeline (one-line commit)
$ git commit -m "docs: update README [ci skip]"

# run pipeline (multi-line commit) as it is not part of the subject
$ git commit -m "docs: update README

- add section installation
- [ci skip]"

The Jenkins build status will be set to NOT_BUILT, the Bitbucket build status to SUCCESSFUL (as there is no "skipped" state). The pipeline will start to execute initially, but abort before launching any agent nodes or starting any of the stages defined in the Jenkinsfile.

Stages

Each built-in stage (like odsComponentStageScanWithSonar) takes two arguments:

  • context (required, this is the pipeline context)

  • config (optional, a map of configuration options)

Example:

odsComponentStageScanWithSonar(context, [branch: 'production'])

odsComponentFindOpenShiftImageOrElse

Checks if an image for the current commit exists already, otherwise executes the given closure.

Example:

odsComponentFindOpenShiftImageOrElse(context) {
  stage('Build') {
    // custom stage to build your application package
  }
  odsComponentStageBuildOpenShiftImage(context)
}

The step can be customized using the options resourceName and imageTag.

Using this step in your Jenkinsfile allows you to avoid building a container image for the same Git commit multiple times, reducing build times and increasing reliability as you can promote the exact same image from one environment to another.

Keep in mind that image lookup works by finding an image tagged with the current Git commit. If you merge a branch into another using a merge commit, the current Git commit SHA will differ from the previously built image tag, even if the actual contents of the repository are the same. To ensure image importing kicks in, use the --ff-only option on git merge (this can also be enabled for pull requests in Bitbucket under "Merge strategies"). There are a few consequences when doing so, which should be kept in mind:

  • No merge commit is created, which has the downside that you do not see when a PR was merged, and that the merge commit is a convenient way to find the associated PR. Further, if the latest commit on a branch which you want to merge contains [ci skip], beware that the build on the target branch will also be skipped. That siad, having no merge commit has the upside that your Git history is not polluted by merge commits.

  • Enforcing a fast-forward merge prevents you from merging a branch which is not up-to-date with the target branch. This has the downside that before merging, you may need to rebase your branch or merge the target branch into your branch if someone else updated the target branch in the meantime. While this may cause extra work, it has the upside that you cannot accidentally break the target branch (e.g. tests on your branch may work based on the outdated target branch, but fail after the merge).

In summary, using git merge --ff-only provides safety, a clean history and allows to promote the exact same image between environments.

odsComponentStageScanWithSonar

The "SonarQube Analysis" stage scans your source code and reports findings to SonarQube. The configuration of the scan happens via the sonar-project.properties file in the repository being built.

If your SonarQube server edition allows to scan multiple branches (any commercial edition does), then this stage will automatically decorate pull requests in Bitbucket with feedback from SonarQube (if the PR already exists at the time of the Jenkins pipeline run).

In debug mode, the sonar-scanner binary is started with the -X flag.

If no sonar.projectVersion is specified in sonar-project.properties, it is set to the shortened Git SHA.

Options

Option Description

analyzePullRequests
boolean

Whether to analyze pull requests and decorate them in Bitbucket. Turned on by default, however a scan is only performed if the branch property allows it.

branch
String

Branch to scan. Example: 'master'. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

branches
List<String>

Branches to scan. Example: ['master', 'develop']. Next to exact matches, it also supports prefixes (e.g. feature/) and \ all branches (). Defaults to master for the community edition of SonarQube, and for all other editions (which are capable of handling multiple branches).

longLivedBranches
List<String>

Branch(es) for which no PR analysis should be performed. If not set, it will be extracted from branchToEnvironmentMapping of the context.

requireQualityGatePass
boolean

Whether to fail the build if the quality gate defined in the SonarQube project is not reached. Defaults to false.

resourceName
String

Name of BuildConfig/ImageStream of the image that we want to scan (defaults to context.componentId). BuildOpenShiftImageStage puts the imageRef into a map with the resourceName as key. In order to be able to receive the imageRef for scanning, the resourceName needs to be the same as in BuildOpenShiftImageStage.

odsComponentStageScanWithAqua

The "Aqua Security Scan" stage scans an image that was previously built in that same pipeline run.

As a result, a Bitbucket Code Insight entry is added to the git commit (in Bitbucket) that basically contains a link to the scan result on the Aqua platform. The Bitbucket Code Insight entry can be seen in a pull request. The pull request in Bitbucket shows the Code Insight of the latest commit of the PR.

To get started, make sure you have a ConfigMap in OpenDevStack project namespace (usually ods) in OpenShift that has these fields:

...
metadata:
  name: aqua
...
data:
  registry: <registry-name-in-aqua-platform>
  secretName: <secret-name-of-aqua-user-credentials>
  url: <aqua-platform-url>
  enabled: <true/false>
  nexusRepository: <name-repository-in-nexus-to-store-the-results>
  alertEmails: <emails-to-send-notifications>

  1. registry: Refers to a name for the image registry given in the Aqua platform by an Aqua platform admin.

  2. secretName: Name of a Secret that contains the credentials of the Aqua platform user that is used for executing the scan. That user needs to have scanner rights. This field is optional, if the property doesn’t exists the system will use the credential 'cd-user-with-password'.

  3. url: Base URL of the Aqua platform (including scheme).

  4. enabled: If true, the scan always occur in all projects. False to disable the scan.

  5. nexusRepository: Name of the repository in Nexus instance to store the results of analysis in HTML format.

  6. alertEmails: Optional field. It contains the emails splitted by ',' to send error notifications regarding with Aqua analysis (misconfigurations, etc…​). The mail server must be configured in Jenkins to send the emails.

Is possible to disable the analysis at project level. for that is necessary to add in the ConfigMap new properties e.g. like this:

...
metadata:
  name: aqua
...
data:
  registry: <registry-name-in-aqua-platform>
  secretName: <secret-name-of-aqua-user-credentials>
  url: <aqua-platform-url>
  enabled: <true/false>
  nexusRepository: <name-repository-in-nexus-to-store-the-results>
  alertEmails: <emails-to-send-notifications>
  project.key1.enabled: <false>
  project.key2.enabled: <false>

  1. project.key1.enabled: Property to indicate that key1 (being key1 the key of the project) has the aqua analysis disabled.

  2. project.key2.enabled: The same but for key2 project.

odsComponentStageScanWithTrivy

The "Trivy Security Scan" stage scans the filesystem of the cloned repository using Trivy and generates a SBOM report, with CycloneDX format by default. Check Trivy supported formats here.

As a result, a Bitbucket Code Insight entry is added to the git commit (in Bitbucket) that basically contains a link to the scan report stored in Nexus. The Bitbucket Code Insight entry can be seen in a pull request. The pull request in Bitbucket shows the Code Insight of the latest commit of the PR.

To get started, edit your Jenkinsfile and add the Trivy stage:

) { context ->
    ...
    odsComponentStageScanWithTrivy(context)
    ...
}

Options

Option Description

additionalFlags
List<String>

Additional flags for the Trivy CLI. Please refer to the official Trivy CLI reference for possible options and don’t forget to take the CLI version of your ODS installation into account. The value of additionalFlags must be a list in which the entries have the official flag name and a possible value. Example: ['--debug', '--timeout=10m']

branch
String

Branch to run stage for. Example: 'master'. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

branches
List<String>

Branches to run stage for. Example: ['master', 'develop']. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

format
String

Set the format for the generated report. Defaults to cyclonedx.

nexusDataBaseRepository
String

Name of the Nexus repository used to proxy the location of the database of vulnerabilities located in GitHub. Defaults to docker-group-ods.

nexusReportRepository
String

Name of the Nexus repository where the scan report will be stored. Defaults to leva-documentation.

reportFile
String

Name of the file that will be archived in Jenkins and uploaded in Nexus. Defaults to trivy-sbom.json.

resourceName
String

Name of component that we want to scan. Defaults to context.componentId.

scanners
String

Comma-separated list of what security issues to detect. Defaults to vuln,config,secret,license.

vulType
String

Comma-separated list of vulnerability types to scan. Defaults to os,library.

odsComponentStageScanWithSnyk

The "Snyk Security Scan" stage performs two tasks:

  1. It uploads your 3rd party dependencies including their licenses for monitoring. Snyk will then notify developers about new vulnerabilities per email once they are reported to the Snyk Vulnerability Database.

  2. It analyses your 3rd party dependencies including their licenses and breaks the build if vulnerable versions are found.

To get started, setup an organisation in snyk.io with exactly the same name as your ODS project name. Under "Settings", create a service account for this organisation and make a note of the displayed token. Edit your Jenkinsfile and add the Snyk stage:

) { context ->
    ...
    odsComponentStageScanWithSnyk(context, [snykAuthenticationCode: <your token>])
    ...
}

It is recommended to read your authentication token dynamically, e.g. from an environment variable or a credential in your Jenkins master.

Options

Option Description

additionalFlags
List<String>

Additional flags for the Snyk CLI. Please refer to the official Snyk CLI reference for possible options and don’t forget to take the CLI version of your ODS installation into account. The value of additionalFlags must be a list in which the entries have the official flag name and a possible value. Example: ['--all-sub-projects', '--show-vulnerable-paths=all']

branch
String

Branch to run stage for. Example: 'master'. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

branches
List<String>

Branches to run stage for. Example: ['master', 'develop']. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

buildFile
String

Build file from which to gather dependency information. Defaults to build.gradle.

failOnVulnerabilities
boolean

Whether to fail the build when vulnerabilities are found. Defaults to true.

organisation
String

Name of the Snyk organisation. Default to context.projectId.

projectName
String

Name of the Snyk project name. Default to context.componentId.

severityThreshold
String

Severity threshold for failing. If any found vulnerability has a severity equal or higher to the threshold, the snyk test will return with a failure status. Possible values are low, medium, high. Defaults to low.

snykAuthenticationCode
String

Required! Authentication token of a service account within your organisation.

odsComponentStageBuildOpenShiftImage

Triggers (and follows) a build in the BuildConfig related to the repository being built.

The resulting image is tagged with context.shortGitCommit.

Options

Option Description

branch
String

Branch to run stage for. Example: 'master'. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

branches
List<String>

Branches to run stage for. Example: ['master', 'develop']. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

buildArgs
Map<String, String>

Pass build arguments to the image build process.

buildTimeoutMinutes
Integer

Timeout of build (defaults to 15 minutes).

buildTimeoutRetries
Integer

Adjust retries to wait for the build pod status (defaults to 5).

dockerDir
String

Docker context directory (defaults to docker).

extensionImageLabels
Map<String, String>

Extra image labels added into imageLabels

imageLabels
Map<String, String>

Pass labels which should be added on the image. Each label will be prefixed with ext..

imageTag
String

Image tag to apply (defaults to context.shortGitCommit).

resourceName
String

Name of BuildConfig/ImageStream to use (defaults to context.componentId).

odsComponentStageImportOpenShiftImage

Imports an image from another namespace.

By default, the source image is identified using the commit which triggered the pipeline run.

Options

Option Description

branch
String

Branch to run stage for. Example: 'master'. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

branches
List<String>

Branches to run stage for. Example: ['master', 'develop']. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

imagePullerSecret
String

Name of image-puller secret (optional, used when pulling images from an external source cluster).

resourceName
String

Name of BuildConfig/ImageStream to use (defaults to context.componentId).

sourceProject
String

OpenShift project from which to import the image identified by resourceName.

sourceTag
String

Image tag to look for in the sourceProject (defaults to context.shortGitCommit).

targetTag
String

Image tag to apply to the imported image in the target project (defaults to sourceTag).

odsComponentStageRolloutOpenShiftDeployment

Rolls out the current resources as defined in the component.

Without any configuration the stage tries to guess what a user expects. If the component contains a directory name chart, a Helm deployment is assumed. If the component contains a directory name openshift, a Tailor deployment is assumed. If neither exists a Tailor deployment is assumed.

Helm

Triggers a release or update of an release with Helm.

The stage will use the helm command to trigger the release. The command will be executed in the directory referenced by chartDir. If the directory does not exist, the stage will fail.

The images used in the deployment will not be tagged or otherwise modified.

HELM_DIFF_IGNORE_UNKNOWN_FLAGS=true helm -n play-dev secrets diff upgrade \
  --install --atomic --force \
  -f values.yaml \
  --set registry=registry.example.com \
  --set componentId=example-helm-chart \
  --set imageNamespace=example-dev \
  --set imageTag=deadbeef69cafebabe \
  --no-color --three-way-merge --normalize-manifests \
  example-release . || true

# run the upgrade
helm -n play-dev secrets upgrade \
  --install --atomic --force \
  -f values.yaml \
  --set registry=registry.example.com \
  --set componentId=example-helm-chart \
  --set imageNamespace=play-dev \
  --set imageTag=deadbeef69cafebabe \
  example-release .

Tailor

Triggers (and follows) a rollout of the DeploymentConfig related to the repository being built.

It achieves this by tagging the image built in odsComponentStageBuildOpenShiftImage with latest. This might already trigger a rollout based on an existing ImageTrigger. If none is set, the stage will start a manual rollout.

If the directory referenced by openshiftDir exists, the templates in there will be applied using Tailor. In this case, it is recommended to remove any image triggers to avoid duplicate rollouts (one when configuration changes due to a config trigger and one when the image is tagged to latest). In addition to the configuration options below, one can use e.g. a Tailorfile to adjust the behaviour of Tailor as needed.

Options

Option Description

branch
String

Branch to run stage for. Example: 'master'. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

branches
List<String>

Branches to run stage for. Example: ['master', 'develop']. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

chartDir
String

Directory of Helm chart (defaults to chart).

deployTimeoutMinutes
Integer

Adjust timeout of rollout (defaults to 5 minutes). Caution: This needs to be aligned with the deployment strategy timeout (timeoutSeconds) and the readiness probe timeouts (initialDelaySeconds + failureThreshold * periodSeconds).

deployTimeoutRetries
Integer

Adjust retries to wait for the pod during a rollout (defaults to 5).

helmAdditionalFlags
List<String>

List of additional flags to be passed verbatim to to helm upgrade (empty by default). Only relevant if the directory referenced by chartDir exists.

helmDefaultFlags
List<String>

List of default flags to be passed verbatim to to helm upgrade (defaults to ['--install', '--atomic']). Typically these should not be modified - if you want to pass more flags, use helmAdditionalFlags instead. Only relevant if the directory referenced by chartDir exists.

helmDiff
boolean

Whether to show diff explaining changes to the release before running helm upgrade (true by default). Only relevant if the directory referenced by chartDir exists.

helmEnvBasedValuesFiles
List<String>

List of paths to values files (empty by default). Only relevant if the directory referenced by chartDir exists. These must contain a suffix called '.env.yml' - which will be replaced during rollout and deployment, and then added to helmValueFiles

Passing a string literal of 'values.env.yaml' will be expanded to their respective environments.

For example: 'values.env.yaml' will become 'values.dev.yaml', 'values.test.yaml' or 'values.prod.yaml'. That means creating the usual files that are named after their respective environment are parsed as usual.

helmPrivateKeyCredentialsId
String

Credentials name of the private key used by helm-secrets (defaults to ${context.cdProject}-helm-private-key). The fingerprint must match the one specified in .sops.yaml. Only relevant if the directory referenced by chartDir exists.

helmReleaseName
String

Name of the Helm release (defaults to context.componentId). Change this value if you want to install separate instances of the Helm chart in the same namespace. In that case, make sure to use {{ .Release.Name }} in resource names to avoid conflicts. Only relevant if the directory referenced by chartDir exists.

helmValues
Map<String, String>

Key/value pairs to pass as values (by default, the key imageTag is set to the config option imageTag). Only relevant if the directory referenced by chartDir exists.

helmValuesFiles
List<String>

List of paths to values files (empty by default). Only relevant if the directory referenced by chartDir exists.

imageTag
String

Image tag on which to apply the latest tag (defaults to context.shortGitCommit).

openshiftDir
String

Directory with OpenShift templates (defaults to openshift).

selector
String

Selector scope used to determine which resources are part of a component (defaults to context.selector).

tailorExclude
String

Resource kind exclusion used by Tailor (defaults to bc,is). Only relevant if the directory referenced by openshiftDir exists.

tailorParamFile
String

Path to Tailor parameter file (defaults to none). Only relevant if the directory referenced by openshiftDir exists.

tailorParams
List<String>

Additional parameters to pass to Tailor (defaults to []). Only relevant if the directory referenced by openshiftDir exists.

tailorPreserve
List<String>

Paths to preserve in the live configuration (defaults to []). Only relevant if the directory referenced by openshiftDir exists.

tailorPrivateKeyCredentialsId
String

Credentials name of the private key used by Tailor (defaults to ${context.cdProject}-tailor-private-key). Only relevant if the directory referenced by openshiftDir exists.

tailorSelector
String

Selector scope used by Tailor (defaults to config option selector). Only relevant if the directory referenced by openshiftDir exists.

tailorVerify
boolean

Whether Tailor verifies the live configuration against the desired state after application (defaults to true). Only relevant if the directory referenced by openshiftDir exists.

Notable Differences between tailor and helm deployments

When tailor does the rollout, all the created or updated OpenShift resources are automatically labeled to ease their management. This is in contrast to helm rollouts which rely on the chart providing the desired labels. Add labels either via the chart directly or via supplying them in the values or values files.

Detailed information about the labelling can be found here.

odsComponentStageUploadToNexus

Triggers the upload of an artifact to Nexus. Implementation is based on https://help.sonatype.com/repomanager3/rest-and-integration-api/components-api

Options

Option Description

artifactId
String

For repositoryType=maven2: default is context.componentId

branch
String

Branch to run stage for. Example: 'master'. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

branches
List<String>

Branches to run stage for. Example: ['master', 'develop']. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

distributionFile
String

Filename. Defaults to ${context.componentId}-${context.tagversion}.tar.gz

groupId
String

For repositoryType=maven2: default is the groupId on project level, or in case not set at all org.opendevstack.${context.projectId}

repository
String

Name of the Nexus repository. Defaults to candidates.

repositoryType
String

Type of the Nexus repository. Defaults to maven2.

targetDirectory
String

For repositoryType=raw: default is context.projectId

version
String

For repositoryType=maven2: default is context.tagversion

odsComponentStageCopyImage

Copies a source image into the project.

This is useful to get images into the OpenShift registry so that release manager will accept all images.

The primary intention is for helm charts so that external images can be imported.

Options

Option Description

branch
String

Branch to run stage for. Example: 'master'. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

branches
List<String>

Branches to run stage for. Example: ['master', 'develop']. Next to exact matches, it also supports prefixes (e.g. feature/) and all branches (*).

sourceCredential
String

sourceCredential is the token to use, if any, to access the source registry

sourceImageUrlIncludingRegistry
String

Source image to import

This needs to be in the following format: [REGISTRY/]REPO/IMAGE[:TAG]

tagIntoTargetEnv
Boolean

true will tag the image from the -cd namespace into the targetEnvironment that the pipeline is running for

verifyTLS
Boolean

verifyTLS allows the stage to ignore certificate validation errors.

The default is to verify certificate paths