Provisioning App: Configuration Guide

Quickstarters

Available quickstarters can be provided in the container via /quickstarters/quickstarters.properties. In the context of OpenShift, this file is supplied by the ConfigMap quickstarters.properties. The quickstarters defined there will be displayed to the users in the UI when they modify an existing project.

When the provisioning app is installed for the first time, a default set of quickstarters (from the ods-quickstarters repository) is seeded. As each installation can configure the available quickstarters differently, the quickstarters.properties ConfigMap is not updated automatically when updating ODS.

A quickstarter needs to have two mandatory entries:

quickstarters.properties
jenkinspipeline.quickstarter.<quickstarter-key>.desc=<description>
jenkinspipeline.quickstarter.<quickstarter-key>.repo=<repository>

For example:

quickstarters.properties
jenkinspipeline.quickstarter.be-java-springboot.desc=Backend - SpringBoot/Java
jenkinspipeline.quickstarter.be-java-springboot.repo=ods-quickstarters

This means that in order to provision the quickstarter, the provisioning app will start a Jenkins pipeline based on the Jenkinsfile located in the opendevstack/ods-quickstarters repository, in the folder be-java-springboot. The UI will display the quickstarter as Backend - SpringBoot/Java.

Apart from the required properties, there are also optional properties:

quickstarters.properties
jenkinspipeline.[some-job-name].create-webhook -> whether a default webhook should be created for this repo or not (true by default)
jenkinspipeline.[some-job-name].branch -> a branch differing from ods.git-ref property
jenkinspipeline.[some-job-name].jenkinsfile -> a path to the Jenkinsfile differing from [some-job-name]/Jenkinsfile
When the quickstarter properties are modified, the provisioning app needs to be restarted for the changes to have effect.

Permissions

By default no special permissions are set on either confluence / jira / bitbucket or openshift, only system-wide settings are inherited.

However there is a special knob to tighten security (which can be passed with the project input createpermissionset : boolean) - based on three special groups that need to be provided as part of the API call / from the userinterface.

  1. admin group: admin rights on the generated projects / spaces / repositories

  2. user group: read / write rights on the generated projects / spaces / repositories

  3. readonly group: read rights on the generated projects / spaces / repositories

The configuration for the permission sets are configured:

  1. JIRA Project is provisioned with its own permissionset defined in src/main/resources/permission-templates/jira.permission.all.txt

  2. Confluence Project is provisioned with special permission set defined in src/main/resources/permission-templates/confluence.permission.*

  3. Bitbucket Project is provisioned with tight read & write roles

  4. Openshift Project roles linked to the passed groups (READONLY - view, ADMINGROUP - admin, USERS - edit)

Furthermore if you need to define default permission for openshift (e.g. to setup membership permission for cluster admins) you can add this to your application properties:

jenkinspipeline.create-project.default-project-groups=ADMINGROUP=<MY_CLUSTER_ADMIN_GROUP_NAME>

In case special permissions sets are defined this the default project groups will be appended to the lis of permissions sets.

Project/Space types based on templates

The default jira / confluence project' types are defined in src/main/resources/application.properties - and correspondingly in the config maps

project.template.key.names=default

jira.project.template.key=com.pyxis.greenhopper.jira:gh-scrum-template
jira.project.template.type=software

confluence.blueprint.key=com.atlassian.confluence.plugins.confluence-software-project:sp-space-blueprint

To add a new template - copy, and add your config, based on a new <project-template-name>

jira.project.template.key.<project-template-name>=com.pyxis.greenhopper.jira:gh-scrum-template
jira.project.template.type.<project-template-name>=software

# optional, can stay as is
confluence.blueprint.key.<project-template-name>=com.atlassian.confluence.plugins.confluence-software-project:sp-space-blueprint

and add the new from above to the existing property project.template.key.names

# list of templates surfaced to the UI and API
project.template.key.names=default,<project-template-name>

Custom permissions and group-to-project-role mappings

Custom permission configuration makes possible to configure each Project type template with a specific permission-scheme-id on project creation.

Any new Jira project of a specific project type will be then set with the same permission-schemed-id.

Additionally, mappings of project roles to the special groups (defined above: admin, user and readonly group) have to be configured, so that the special groups are added to the jira project roles of the new project.

These capabilities make possible for Jira Admin to define a main permission scheme that will be "reused" for every new project. Another big benefit is that release them from the manual task of mapping the special groups to the permission scheme project roles.

Project type custom permissions scheme id configuration

To configure a project-template-name with a specific permission-schemed-id you need to add these properties to your configuration:

jira.project-templates.<project-template-name>.name=UTest project template
jira.project-templates.<project-template-name>.permission-scheme-id=<PERMISSION_SCHEME_ID>
if the permission-scheme-id is configured you will have to provide group-to-project-role mapping configuration.

Project type group-to-project-role mappings

If a permission-schemd-id is defined, then the following project-to-* properties have to be configured. Otherwise the provisioning app will fail to start.

Like the permission-scheme-id the value of these properties has to be in this case an existant project role id in Jira, that you will need to get from your Jira server in advance.

jira.project-project-template-name.<>.role-mapping.project-role-for-admin-group=<ROLE_ID>
jira.project-project-template-name.<>.role-mapping.project-role-for-user-group=<ROLE_ID>
jira.project-project-template-name.<>.role-mapping.project-role-for-readonly-group=<ROLE_ID>

Overwriting custom permissions scheme id and group-to-project-role mappings per API call

Another way to set a permission-scheme-id with the corresponding project-to-role-* mappings is by API call. You can define following properties in the payload of the create project API call:

{
    ...
    "specialPermissionSchemeId": "<PERMISSION_SCHEME_ID>"
    "projectRoleForAdminGroup": "<ROLE_ID>",
    "projectRoleForUserGroup": "<ROLE_ID>",
    "projectRoleForReadonlyGroup": "<ROLE_ID>",
    ...
}

A given <project-template-name> configuration will be overwritten by these payload properties.

You will find more details about the provisioning app REST API in this section: Consuming REST APIs via curl.

If no permission-scheme-id with the corresponding project-to-role-* mappings are provided neither by configuration nor in the create project payload, then the default behaviour will be applied, which in this case will create a new permission scheme as explained in the section Permissions.

Add Webhook Proxy URL to jira project properties based on project type

It is possible to configure the Provisioning App to add to jira project the Webhook Proxy URL as project property. Jira provides an REST API for this purpose (Jira Properties API)

This functionality can be configured for each project type. To enable this you will need to:

  • Enable this capability for a given project type add the a property like:

jira.project.template.add-webhook-proxy-url-as-project-property.<project-template-name>=true
  • Define the jira endpoint as template by adding this property:

jira.project.template.webhook-proxy-url-endpoint-template.<project-template-name>=/api/2/project/%PROJECT_KEY%/properties/WEBHOOK_PROXY.URL

These 2 template keys %PROJECT_KEY% and %PROPERTY_VALUE% can be defined and will be replaced will real values.

  • Define the jira endpoint payload as template that will be added to the set jira property endpoint call

jira.project.template.webhook-proxy-url-payload-template.<project-template-name>={\"WEBHOOK_PROXY.URL\", \"%PROPERTY_VALUE%\"}

For the payload template also these 2 template keys %PROJECT_KEY% and %PROPERTY_VALUE% can be defined and will be replaced will real values.

Error Handling

Up to (and including) v1.1.x when provisioning failed, corrupt and inconsistent states where left in the bugtracker system, bitbucket etc. which had do be cleaned up manually based on logs. This is rectified and a the new default behavior is to see every post to the API as atomic unit of work, which in case of failure is tried to be cleaned up (alike functional rollback). This behavior can be turned off by specifying the new property provision.cleanup.incomplete.projects and setting it to false.

Authentication Configuration

There are to separate authentication options to authenticated a user for the provisioning app.

  • CROWD (default)

  • OAUTH2

  • Basic Auth

Note that the current OAUTH2 implementation is only used for authentication the user to the provisioning app. To authentication that is used for the REST - API calls of Atlassian Crowd, Jira, Confluence and Bitbucket is done eighter via the logged in uses credentials (user name and password) or via the technical users, that are configured in the used spring boot profile.

Authentication Crowd Configuration

application.properties
provision.auth.provider=crowd (1)
spring.profiles.active=crowd (2)
1 configures crowd authentication provider
2 include crowd profile per default.

The crowd specific configuration is done in the included profile crowd, see property documentation inside the profile file application-crowd.properties. The provided example configuration is appropriate for a locally installed OpenDevStack environment.

Authentication OAUTH2 Configuration

An example of plain oauth2 configuration is given in spring boot profile application-oauth2.properties. The provided example configuration is appropriate for a locally installed OpenDevStack environment, when the idmanager vagrant box is used.

application-oauth2.properties
provision.auth.provider=oauth2 (1)

idmanager.url=http://192.168.56.32:8080 (2)
idmanager.realm=provisioning-app (3)

oauth2.user.roles.jsonpointerexpression=/claims/roles (4)

(5)
spring.security.oauth2.client.registration.keycloak.client-id=ods-provisioning-app
spring.security.oauth2.client.registration.keycloak.client-secret=put-your-secret-here
spring.security.oauth2.client.registration.keycloak.clientName=ods-provisioning-app
spring.security.oauth2.client.registration.keycloak.authorization-grant-type=authorization_code
spring.security.oauth2.client.registration.keycloak.redirectUri={baseUrl}/login/oauth2/code/{registrationId}
spring.security.oauth2.client.registration.keycloak.scope=openid

(6)
spring.security.oauth2.client.provider.keycloak.authorization-uri=${idmanager.url}/auth/realms/${idmanager.realm}-app/protocol/openid-connect/auth
spring.security.oauth2.client.provider.keycloak.token-uri=${idmanager.url}/auth/realms/${idmanager.realm}/protocol/openid-connect/token
spring.security.oauth2.client.provider.keycloak.user-info-uri=${idmanager.url}/auth/realms/${idmanager.realm}/protocol/openid-connect/userinfo
spring.security.oauth2.client.provider.keycloak.jwk-set-uri=${idmanager.url}/auth/realms/${idmanager.realm}/protocol/openid-connect/certs
spring.security.oauth2.client.provider.keycloak.user-name-attribute=preferred_username
1 configures oauth2 authentication provider
2 URL to idmanager. The value defaults to opendevstack idmanager box with keycloak installation
3 Name of realm that is used
4 The application reads the user roles from the claim that is inside the oauth2 ID-Token. The property oauth2.user.roles.jsonpointerexpression is a JsonPointer - Expression that defines a path to the roles that are extracted from the id token. Details regarding pointer expression can be found at Jackson-core JsonPointer Documentation
5 Defines the OAUTH2 client registration properties, in particular the client name and client secret. See OAUTH2 Spring Boot 2.x Property Mappings for details.
6 Defines the OAUTH2 client.provider properties. These properties correspond to the well-known OAUTH2-URIs. In case of keycloak, this URIs can be read out using the .well-known/openid-configuration-link

Alternatively if your identity provider is Azure AD, a configuration example is given in application-azure.properties

application-azure.properties
provision.auth.provider=oauth2 (1)
provision.auth.provider.oauth2.user-info-uri=userInfo (2)

(3)
# Application ID (also called Client ID)
spring.security.oauth2.client.registration.azure.client-id=<CLIENT_ID>>
spring.security.oauth2.client.registration.azure.client-secret=<CLIENT_SECRET>

# It's suggested the logged in user should at least belong to one of the below groups
# If not, the logged in user will not be able to access any authorization controller rest APIs
azure.activedirectory.user-group.allowed-groups=opendevstack-administrators,opendevstack-users (4)
azure.activedirectory.environment=global-v2-graph
azure.activedirectory.user-group.key=@odata.type
azure.activedirectory.user-group.value=#microsoft.graph.group
azure.activedirectory.user-group.object-id-key=id
azure.activedirectory.tenant-id=<TENANT_ID> (5)

oauth2.user.roles.jsonpointerexpression=/claims/roles (6)
oauth2.user.use-email-claim-as-username=true (7)

(8)
idmanager.url=https://login.microsoftonline.com
idmanager.realm=${spring.security.oauth2.client.registration.azure.client-id}

idmanager.disable-logout-from-idm=true (9)
1 configures oauth2 authentication provider
2 configure user info uri
3 registers in spring security azure oauth2 client id and secret
4 configure allow groups
5 register azure ad tenant
6 the application reads the user roles from the claim that is inside the oauth2 ID-Token. The property oauth2.user.roles.jsonpointerexpression is a JsonPointer - Expression that defines a path to the roles that are extracted from the id token. Details regarding pointer expression can be found at Jackson-core JsonPointer Documentation
7 configure to use email claim as username
8 configure name of the ProvApp realm
9 instruct ProvApp to not logout from identity management provider

Basic Auth authentication

This option can be enabled to activate basic auth as additional authentication when using crowd or oauth2 as authentication provider.

To enable basic auth this properties needs to be configured:

application-azure.properties
provision.auth.basic-auth.enabled=true

The basic auth authentication needs to connect to an identity manager to authenticate users. Currently only crowd is supported for this purpouse. If you have’ve chosen to use oauth2 as provider you can even enable basic auth to connect to crowd server. For that you will need to add these properties to your configuration:

# crowd properties (needed for basic auth)
crowd.local.directory=~/dev/temp
crowd.application.name=<APPLICATION_NAME>
crowd.application.password=<PASSWORD>
crowd.server.url=<CROWD_HOST>
crowd.cookie.domain=<COOKIE_DOMAIN>

Authentication to third party apps via technical users

The rest api calles use HTTP basic access authentication to communicate with Jira, Confluence and Bitbucket. The used credentials are read from a pair of properties. For Example, bitbucket.admin_password and bitbucket.admin_user properties are used for Bitbucket, confluence.admin_user and confluence.admin_password are used for Confluence, etc.

application-oauth2.properties
# configure technical user for bitbucket. Do not authenticate via oauth2, since not implemented.
bitbucket.admin_password=bitbucket_admin
bitbucket.admin_user=bitbucket_admin

# configure technical user for confluence. Do not authenticate via oauth2, since not implemented.
confluence.admin_password=confluence_admin
confluence.admin_user=confluence_admin

# configure technical user for jira. Do not authenticate via oauth2, since not implemented.
jira.admin_password=jira_admin
jira.admin_user=jira_admin
Note: if the pair of properties is not defined for a third party tool, the logged in user’s credentials are used to authenticate against the application. The credentials are read by caling the method getUserName and getUserPassword from IODSAuthnzAdapter]. See also implementation of org.opendevstack.provision.services.BaseServiceAdapter#authenticatedCall()

If you need to display a disclaimer in the front-end you can add this property to the application properties:

provision.ui.disclaimer=<DISCLAIMER_TEXT>
this property is not supported yet in the single page front-end.

FAQ

  1. Where is the provision app deployed?

    1. the provision application is deployed on openshift, in both prov-dev and prov-test. prov-dev is the development environment in case you want to change / enhance the application, while the production version of the application is deployed in prov-test. The URL to get to the provision application, is defined thru a route. Ít’s https://prov-app-test..

  2. Where do I find the logs, if something went wrong?

    1. Within the Openshift pod of the provision app (in projectdev/test, namely in /opt/provision/history/logs a logfile is created per project)

  3. Where is the real configuration of the provision application?

    1. The base configuration in the the application.properties in the codebase, the setup specific one is in a config map deployed within the prov-dev/test project.

  4. What is the default permission schema in Jira?

    1. the default permission schema in Jira is named Default Permission Scheme

  5. Which role is missing in Jira default permission schema if the project creator cannot access it?

    1. The default permission schema named Default Permission Scheme needs the role owner to be added to permission Browser Projects. Otherwise the project creator (role owner) will not be able to access the project.