GitHub actions and workflows are great. We use them to deploy a web app to different cluster environments and I want to show you how we did it. Hopefully, this helps to simplify your deployment process as well. I’ve also written a companion article that describes our GitHub workflows for continuous integration.
Here, we’ll focus on our continuous deployment process:
Workflow Overview
Our continuous deployment workflow has four jobs. It includes the same jobs that we had in our continuous integration workflow, as well as two extra jobs:
To understand our deployment workflow, you need to know a bit about our release process.
As mentioned in the preceding article, we set up these workflows while building a web app for a major property tech startup.
When creating a release in GitHub, we use the prerelease flag to indicate that the release is not ready for production. In this case, the container is either deployed to a development or staging cluster depending on what branch was used to create the release.
Development: When we’re in the process of developing, we push everything to a branch and create a release from that branch. We set the prerelease flag and give it a descriptive tag with the prefix “rc-” (release candidate).
Staging: When we’re reading to push to staging for proper QA, we merge to master and create another release. We select the master branch but keep the prerelease flag. We add a tag that indicates the intended version, but we keep the prefix “rc-”.
Production: When we deploy to production we add a tag with the prefix “r-” and the “pre-release” flag is no longer selected.
The “r-” and “rc-” tag prefixes enable us to easily distinguish between “real” releases and release candidates when reviewing the release history. As you’ll soon see, we automatically validate tag prefixes when deploying to a production cluster.
Now that you understand our release process, the deployment logic in the workflow file will make a lot more sense. So let’s go through what we did to create it. By the way, I’m assuming you’ve created a workflow file before. If you haven’t, check out my first article on creating a workflow file for CI.
I’ll be using the workflow file for our backend app as an example.
Define the triggering event
We wanted our deployment workflow to kick in whenever someone created a release in GitHub so we updated our workflow file as follows.
name: CD
on:
release:
types: [created]
For more information on the trigger syntax, see GitHub’s reference doc on events.
Define Jobs
As mentioned in my introduction, we wanted to validate the GitHub tags and test the app, take a snapshot of it, and push it as a Docker image in our container registry (just in case we needed to roll back to an earlier iteration of the app). Finally, we wanted to deploy the image to a Kubernetes cluster.
For this purpose, we defined these four jobs:
If any one of these jobs fails, the whole workflow is terminated. I won’t go too much into job 2 and job 3 because they also exist in our CI workflow which I’ve covered in a companion article.
Job 1 is interesting because we’re considering an action that we’ve created ourselves. Also, we store the action in a private repository which poses its own challenges as you’ll soon see. So let’s get into it:
We wanted to easily distinguish different release types on our releases page — so we defined the simple tagging rules that I described previously. But rules are no good unless you can enforce them.
Creating a custom action
To check that people are tagging releases correctly, we created a custom action. GitHub actions are essentially small predefined scripts that execute one specific task. There are plenty of user-contributed actions on the Github marketplace, but in this case, we needed to create our own.
GitHub supports two types of action: an action that runs as a JavaScript, or one that runs in a Docker container.
We set up one that runs in a Docker container since that’s what we’re more familiar with. Our action lives in its own private repo with the following file structure.
The most important files are the action metadata (“action.yaml”) and the shell script (“entrypoint.sh”).
The actions.yaml file defines the metadata for the action, according to the metadata syntax.
name: 'Validate tags'
author: 'Hugobert Humperdinck'
description: 'Validate release/pre-release tags'
inputs:
prerelease:
description: 'Tag is prerelease'
required: true
runs:
using: 'docker'
image: 'Dockerfile'
args:
- ${{ inputs.prerelease }}
It also defines the arguments that the action takes. In our action, we depend on the value of the prerelease flag for our validation logic, so we define it as an input for the action.
The entrypoint.sh is the shell script to run in the Docker container. It includes all of our validation logic.
#!/bin/bash
set -e
set -o pipefail
echo "Start validation $1"
BRANCH=$(git branch -r --contains ${GITHUB_SHA} | grep "")
RELEASE_VERSION=$(echo ${GITHUB_REF} | sed -e "s/refs\/tags\///g" | sed -e "s/\//-/g")
MASTER_BRANCH_NAME='origin/master'
RELEASE_PREFIX='r-'
if [[ "${INPUT_PRERELEASE}" != true ]] && [[ "$BRANCH" == *"$MASTER_BRANCH_NAME"* ]] && [[ "$RELEASE_VERSION" == "$RELEASE_PREFIX"* ]]; then
echo "Release tag validation succeeded!"
exit 0
elif [[ "${INPUT_PRERELEASE}" == true ]]; then
echo "Pre-Release tag validation succeeded!"
exit 0
else
echo "Tag validation failed!"
exit 1
fi
Here’s what the script does.
Set environment variables
We’re using the built-in GitHub environment variables GITHUB_REF and GITHUB_SHA to determine the following variables:
Validate Release Tag
We want the tags to be formatted a certain way but we only care about tags for production releases.
This logic could obviously be improved. We could check the prerelease flag in job definition, but my main goal is to show you how a basic action is structured.
Normally, it’s easy to call an action from a workflow file — but that’s assuming the action is in a public repo. Our customer didn’t want to open up their source files for the world to see, so we needed to store our action in a private repository.
Initially, this was a problem. It was difficult to check out a private repository so we couldn’t use an action stored there. I’m telling you this because we’re still in the process of updating our workflow since GitHub recently fixed this problem.
Originally, checking out other repos was simple. When you wanted to clone your working branch to the virtual machine, you called the public action “actions/checkout” without any arguments. You could also pass it extra arguments to check out other public repositories.
But version 1 of the checkout action did not work so well with private repositories. It could check out the repository, but did not pass the path to the next workflow step. If you checked out a repo to the folder “my-magic-actions”, the workflow could not see it.
Luckily, version 2 of the checkout action fixes this issue and it supports SSH for accessing private repositories.
Now, to use a private action, we can update our workflow file to check out our private actions repo like this:
jobs:
validate-release-name:
name: Validate release name
runs-on: 'ubuntu-latest'
steps:
- name: Checkout working branch
uses: actions/checkout@v2
- name: Checkout private actions repo
uses: actions/checkout@v2
with:
repository: acme/private-actions
token: ${{ secrets.GitHub_PAT }} # `GitHub_PAT` is a secret that contains your PAT
path: private-actions
In the job steps, we first call the checkout action without arguments to get the working branch for our application.
Then, we call the “checkout” action again to check out another private repo — the one that contains our action.
For more information on how the Checkout action works, so the checkout README.
After all that, we can finally call our private action with the prerelease flag as an argument.
- name: Validate release tag
uses: private-actions/validate-tag-action
with:
prerelease: 'github.event.release.prerelease'
We get the prerelease flag by using the GitHub context syntax.
These are the same tests that we use in our continuous integration workflow which I have already covered in a companion article.
Again, I already covered this job in my companion article, but there is one small difference this time around. We add the tag_names parameter which changes how we tag the Docker image.
- name: Publish Docker Image
uses: elgohr/[email protected]
env:
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
with:
name: ${{ env.DOCKER_IMAGE }}
username: ${{ steps.gcloud.outputs.username }}
password: ${{ steps.gcloud.outputs.password }}
registry: ${{ env.DOCKER_REGISTRY }}
tag_names: true
buildargs: SSH_PRIVATE_KEY
Instead of sticking with the default behaviour, which is to tag the image with the originating branch, we pass the value of the GitHub release tag as our Docker tag. So in the Docker registry, released images look something like this:
Name: 8cd6851d850b Tag: r-3
Again, this enables us to visually distinguish released Docker images from ones that were pushed as part of the continuous integration workflow. As a reminder, pre-release images are tagged with the branch name like this:
Name: 8cd6851d850b Tag: XYZ-123_add_special_field_todo_magic
Assuming the previous job succeeds, we’re ready to push the Docker image to a Kubernetes cluster in our Google Cloud instance.
First, we need to set a few environment variables:
Set up the environment
We updated our workflow file as follows:
deployment:
name: Deploy backend to cluster
runs-on: 'ubuntu-latest'
needs: [docker-image]
steps:
- name: Checkout working branch
uses: actions/checkout@v1
- name: Set Release version
run: |
echo ::set-env name=RELEASE_VERSION::$(echo ${GITHUB_REF} |
sed -e "s/refs\/tags\///g" | sed -e "s/\//-/g")
- name: Cluster env for production
if: "!github.event.release.prerelease"
run: |
echo ::set-env name=CLUSTER_ENV::prod
- name: Cluster env for staging/dev
if: "github.event.release.prerelease"
run: |
BRANCH=$(git branch -r --contains ${GITHUB_SHA} | grep "")
MASTER_BRANCH_NAME='origin/master'
if [[ "$BRANCH" == *"$MASTER_BRANCH_NAME"* ]]; then
echo ::set-env name=CLUSTER_ENV::stag
else
echo ::set-env name=CLUSTER_ENV::dev
fi
- name: Set Cluster credentials
run: |
echo ::set-env name=CLUSTER_NAME::acme-gke-${{ env.CLUSTER_ENV }}
echo ::set-env name=CLUSTER_ZONE::europe-west3-a
echo ::set-env name=PROJECT_NAME::acme-555555
We check out the working branch again (you have to do this for each job), then set RELEASE_VERSION by extracting the tag name from the end of the GITHUB_REF.
Then we need to set all the variables that we’ll use for the “gcloud” command in a subsequent step:
CLUSTER_ENV: We have some simple logic for defining how to do it:
CLUSTER_NAME: We use the CLUSTER_ENV variable to set the suffix for the full name. So it’s either “acme-gke-prod”, “acme-gke-stag”, or “acme-gke-dev”.
The zone and project name are hardcoded.
Install the necessary tools
Next, we need to install the Kubernetes command-line tool and Helm, which makes it easier to install Kubernetes applications.
- name: Install kubectl
run: |
sudo apt-get install kubectl
- name: Install helm
run: |
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
Note that these tools seem to be preinstalled in the standard GitHub virtual machines now. But they weren’t when we set up our workflow file, so I’ll stick to describing what we did.
Deploy the image to a cluster
Finally, we use all the variables that we defined previously to run the gcloud CLI.
- name: Deploy Release on cluster
env:
GCLOUD_KEY: ${{ secrets.GCLOUD_KEY }}
run: |
echo "$GCLOUD_KEY" | base64 --decode > ${HOME}/gcloud.json
gcloud auth activate-service-account --key-file=${HOME}/gcloud.json
gcloud auth configure-docker
gcloud container clusters get-credentials \
${{ env.CLUSTER_NAME }} --zone
${{ env.CLUSTER_ZONE }} --project ${{ env.PROJECT_NAME }}
# install/upgrade helm chart
helm upgrade --install backend ./deploy/helm/backend \
--values ./deploy/helm/backend/env.values.${{ env.CLUSTER_ENV }}.yaml \
--set env_values.image_version=${{ env.RELEASE_VERSION }
First, we get our Google Cloud key from our repository secrets then we give it to the gcloud CLI as well as our cluster and project details.
Then, we use a Helm chart to instgkecoall our back end application on the Kubernetes cluster. The Helm chart for the back end is stored in the backend repo because we prefer to maintain charts as part of the application (more on that in another article).
When installing, we pass Helm the following arguments to override the default settings in the chart:
— values defines the yaml file that contains the environment variables.
For a production release, it’s “env.values.prod.yaml”.
— set overrides a specific variable in the “env.values” yaml file, namely “image_version”.
In the yaml file, it’s set to “latest” but we want it to use our release version such as “r-3”.
And that’s the end of the workflow. Once it’s triggered, it’s easy to monitor the progress and make sure that everything is deployed correctly. For example, here’s a screenshot of the output for one of our other public projects:
As I mentioned in the first part of this two-part series, it was very easy to set these workflows up. The limitation with actions in private repos was a minor irritation, but GitHub is continually improving their built-in actions as this was soon addressed. Plus there is a growing ecosystem of user-contributed actions for every kind of task. Unless a customer wants us to use something else besides GitHub, we’ll be sticking with GitHub CI/CD workflows for future projects.
A Disclosure Note On The Author and Project A
Merlin blogs about developer innovation and new technologies at Project A — a venture capital investor focusing on early-stage startups. Project A provides operational support to its portfolio companies, including developer expertise. As part of the IT team, he covers the highlights of Project A’s work helping startups to evolve into thriving success stories.