As a CI/CD tool, and as part of the “GitOps” capability, the vRealize Automation Code Stream platform has the ability to respond to events within source control endpoints. When new code is pushed to a Git repository, or a new Pull Request is created, Code Stream can be configured to run a pipeline in response. This can be used to manage infrastructure as code, automate image builds or to automate release pipelines using a Blue/Green strategy.
To quote the official documentation:
Code Stream provides a way to trigger a pipeline if a code change occurs in your GitHub, GitLab, or Bitbucket repository. The Git trigger uses a Git endpoint on the branch of the repository that you want to monitor. Code Stream connects to the Git endpoint through a webhook.
As always, you should check the documentation (Cloud, On-premises, Cloud Release Notes, 8.2 Release Notes) for the most up to date supported integrations. At the time of writing the following endpoints are supported with vRealize Automation Code Stream 8.2, and vRealize Automation Cloud:
- GitHub
- GitLab
- Bitbucket
- GitHub Enterprise
- GitLab Enterprise
- Bitbucket Enterprise
There are some considerations when choosing whether to deploy an on-premises version of Git, or use a SaaS solution. As a general rule of thumb, if you want to trigger pipelines based on Git events, you can always use SaaS Git endpoints with vRealize Automation Cloud, and on-premises Git endpoints with vRealize Automation on-premesis. The communications flow between Git and Code Stream goes both ways, depending on the task, so it’s important to understand the scenario:
- To configure a Git endpoint Code Stream must access the Git API to accept the certificate and validate the endpoint
- To auto-configure the webhook on the Git repository, Code Stream must be able to access the Git API (same as 1)
- To trigger pipelines, the Git endpoint must be able to access the Code Stream API
So, if we take point 1 and 2 from above – configuring the endpoint and configuring the webhook, the following chart shows it’s fairly straightforward:
Triggering pipelines from Git events requires Git to have access to the vRealize Automation Code Stream API, which differs – most notably for Code Stream on-premises. Unless the on-premises API was made publicly available (which we do not recommend), SaaS Git integrations cannot access the on-premises API to trigger the Pipeline:
Adding a Git endpoint to Code Stream
Adding a Git endpoint is largely the same between Git solutions, but there are a few small differences.
| Option | Description |
|---|---|
| Project | Select the Project for which the Git Endpoint will be used |
| Type | Select GIT |
| Name | Enter a name for the Endpoint – e.g. “GitLab-Enterprise-FieldDemo” |
| Description | Enter a description for the endpoint – e.g. “Field Demo repository on GitLab Enterprise” |
| Mark restricted | Select whether the resource is restricted (see How do I manage user access and approvals in Code Stream)
|
| Cloud Proxy (SaaS only) | If this endpoint is located on-premises, select a cloud proxy that connects it to Code Stream. Otherwise, leave the setting as Default.
Note: This field will not be visible for vRealize Automation on-premises. |
| Git server type | Select the Git server type |
| Repo URL | Copy the HTTP repository URL from your repository page, then modify it based on the following:
|
| Branch | Enter the branch to use
Note: most are changing defaults from “master” to “main” |
| Authentication type | Enter the authentication type – Password or Private token
I recommend using a Private token where possible, these can typically be revoked and limited in scope more easily than a user’s password. |
Configuring a Pipeline to be triggered by Git Webhook
Technically, you don’t need to do anything to have a pipeline triggered by a Git webhook, but a lot of pipelines will actually need to clone the repository, and/or use the details of the Push or Pull Request. Code Stream can automatically clone the repository into the container running a CI task. In order to do this, you need to configure the pipeline.
Create a new Pipeline from a Blank template and select the Workspace tab:
- Select a Docker endpoint to run your CI containers
- Enter the name and location of your CI container image – I’m using my codestream-ci image from Docker Hub
- Check the box to automatically clone the repository in CI tasks
Next, select the Input tab
- Select the option to auto-inject the Git parameters
Finally, you can use configure a CI task to clone and use the repository:
- Add a Stage, then add a Task
- Configure the task type as CI
- Enter shell commands to execute as part of the task – in this case I’m simply listing the contents of the Git repository to prove that it’s cloned, and I’ll echo the contents of the README.md file in each repo.
Finally, Save the pipeline and enable it through the ACTIONS menu:
Adding a Git webhook
All of the pieces should now be in place to configure and test a Git webhook – the endpoint is added, and the pipeline is in place to be triggered.
Navigate Triggers > Git > Webhooks for Git, then click + NEW WEBHOOK FOR GIT
| Option | Description |
|---|---|
| Project | Select the Project for which the Git Endpoint will be used |
| Name | Enter a name for the webhook – e.g. “GitLab-Enterprise-FieldDemo-Webhook” |
| Description | Enter a description for the webhook – e.g. “Triggers the Field Demo Git Webhook Test pipeline when code is pushed to GitLab-Enterprise-FieldDemo endpoint” |
| Endpoint | Select the Endpoint to use – e.g. “GitLab-Enterprise-FieldDemo” |
| Branch | Select the branch to use – e.g. “master” or “main” |
| Secret token | Generate a secret token to be shared between the Git repository and Code Stream for additional security |
| File Inclusions/Excusions | Configure explicit file inclusions/exclusions, or regex to match |
| Trigger For Git | Select either a Push or Pull Request |
| API Token | Enter an API token for Git to use to trigger the pipeline – this is a Code Stream API token |
| SSL verification | Check to validate the SSL certificate |
| Pipeline | Select the Pipeline to execute – e.g. “Field Demo Git Webhook Test” |
| Execution trigger delay | Select the Project for which the Git Endpoint will be used |
For more detailed instructions, see How do I use the Git trigger in Code Stream to run a pipeline
Validate and Test the Webhook
I can validate the webhook configuration by logging into my Git repository and checking the webhook configuration, for example in GitLab Enterprise, from my Repository page:
- Select Settings
- Webhooks
- Scroll down to the Project Hooks, click Edit
- Validate the URL matches the Code Stream generated URL, and the Secret Token matches the Secret Token generated
Next, make an edit in the repository and push the change – you can do this directly in GitLab/GitHub/Bitbucket, or using a local IDE like VSCode, or just plain git.
Back in the Webhook configuration in Git, check the deliveries:
Switch back to Code Stream to check that the webhook has triggered the pipeline:
And from there we can click on the Execution to view the details. You can see in the screenshot below that the CI task (Task0) ran a list (ls) on the fielddemo folder, which is the cloned Git repository. It then reads (cat) the README.md file, which shows the contents:
Bonus content
Configure GitLab Enterprise to Allow Webhooks
GitLab enterprise must be configured to allow Code Stream to create webhooks:
- As an admin user, click on the Admin Area
- Expand Settings
- Select Network
- Expand Outbound Requests
- Check to allow requests to the local network from web hooks and services
- Add your vRealize Automation appliance IPs, vRealize Automation Cloud Proxy IPs, or your local subnet to the allow list
- Save changes
Next Steps
I hope this provides a useful deep-dive into configuring Git endpoints with webhooks in vRealize Automation Code Stream. The integration with Git and ability to trigger pipelines allows you to operate infrastructure using GitOps principles – whichever flavour of Git you choose! If you want to find out more about vRealize Automation please visit our website, or to learn more about our features, vRealize Automation Code Stream and explore vRealize Automation Cloud get started with a free 45-day trial!
