Build your own CI/CD pipeline with Drone

TL;DR

This is mostly a cookbook style article. If you expect more of a theoretical guide, I suggest you to look at the Official Drone Documentation and the Continuous Integration and Continuous Delivery Book

Background story

In my current company, I’m in charge of the Backend Engineering as well as DevOps side. As you may know, setting up an automated CI/CD pipeline is the prime duty of a DevOps Engineer. I’m no different.

Historically we’re mostly a Jenkins CI driven company just like most other startups. But we’re heavily cloud enthusiast and distributed development company with scalability in mind. And you know, distributed CI/CD pipeline with Jenkins is not impossible, but hard when you reach at a certain level, as jenkins is almost an ancient (and pioneer) CI/CD tool. Sadly we have reached that level a little bit sooner. So we decided to try something else. Something that fits into our paradigm and adheres to our engineering philosophy.

On the development side, we’re mainly a Golang based company, which proves that we Love Go a lot. We try to use as much Go based open source tools for most of our development needs (and they work great) as we can. So we’ve tried couple of CI/CD tools built with Go. We tried Concourse, but it seems too steep to get into the ride. After some mumble jumble (a.k.a initial R&D), we finally reached at Drone’s footprint, an open source CI/CD tool written in Go. At the time of this writing, Drone is quite popular.

Why Drone?

Or may I ask, why not Drone? It’s a purely distributed, cloud-native, DevOps friendly, team oriented and highly scalable open source CI/CD tool. Phewww, mouth full description, right? If you want to know more about Drone, don’t feel shy to head over to the official site.

I’m pretty excited, so let’s dig deep into the rabbit hole.

Setting up a Drone Server

Drone is mostly a production grade CI/CD server. Unlike Jenkins, it won’t work as intended in your local machine if you don’t have a public IP. Drone works best inside cloud VMs, like AWS, DigitalOcean, GCP etc providers VM instances.

Another option is to use Drone’s free Cloud based SaaS, which is fantastic at no cost (and which we’ll use in this article). This way you won’t have to waste time to setup the server when you’re in a hurry to test it. It’s all there waiting for you.

In my future article, I will show you how you can setup a Drone server inside your own private (but public facing) cloud machine for your projects, instead of relying on Drone Cloud.

Impressed? Enough talking.

Now go to https://cloud.drone.io and login with your github account.

It will ask for Github access, since Drone uses Github’s webhook feature. After login, it will sync your repositories.

Now it’s time for the development setup. For this project I have prepared an example git repo. Fork it and clone it into your local machine. It should work fine. It’s a Go based project, but you don’t need to install Go as we’ll not run the project inside our machine.

After we’re done with the development, it’s time to integrate Drone into our development workflow.

Drone integration

Drone uses YML description file for it’s entire pipeline definition. Basically, you write the steps to perform in your pipeline (like fetch from git, build, test, deploy, run etc). Drone will read this file to understand what you want it to perform.

Here’s our .drone.yml file

kind: pipelinename: default

workspace:base: /gopath: src/github.com/yourname/go-drone-hello-world

steps:

• 
  

  
  

  name: testimage: golangcommands:

  • go get
  • go test

• 
  

  
  

  name: buildimage: golangcommands:

  • go get
  • go build -o output
  • chmod +x output

• 
  

  
  

  name: runimage: golangcommands:

  • go get
  • go build -o output
  • chmod +x output
  • ./output

It’s pretty straight forward, right? Let me describe some pieces of it

We defined a pipeline with several steps. I have splitted test, build, run into separate steps just for the sake of this tutorial. If you read the YML file, you can easily understand what it says. It will fetch, build, test our Github repo inside a Docker container (Drone is completely container based, which is awesome). In this case, we’re using the “golang” official docker image for container building. You may guess, you can use your desired language runtime image for your different language projects.

This is the integration part. As we’re using Guithub with Drone, it will automatically start doing it’s job (as described in the YML file) after you commit any changes to your source code (of course inside git repo) and push it to the remote. Github will fire webhook signal then Drone will catch it and do it’s job.

Enough talking, let’s try. Oh by the way, you should click “Activate” button for your desired git repo in the drone cloud UI. This will activate Drone’s job on that repo.

After selecting “Activate”, it will give you a whole bunch of customization options, like YML file name, public/private projects, crone jobs, CI/CD build tags for README.md file etc. Cool!

After you’ve activated the repo, commit your code and push it to remote origin. Keep an eye on the https://cloud.drone.io main interface/dashboard. You will see Drone picks up your repo within a fraction of second after you pushed and then it has started it’s job.

You’ll see a Yellow Clock animated badge that shows Drone is doing it’s job. Click the repo to see what’s happening inside.

As you can see, there are a bunch of builds listed there as I have experimented earlier. You’ll see your commit message as the title of each build phase, so pick your commit message wisely.

Click on the latest build.

It shows all the steps exactly as you defined in the YAML file. It will also show color badges for success or failed steps. All of our steps have been passed. Awesome!

If you click on each of the tasks, you’ll see what Drone did at that time. very cool UI, isn’t it?

That’s all for this article. You have just scratched the surface of Drone and the whole CI/CD concept. There are a lot to learn from here. Feel free to explore Drone’s documentation, CI/CD books and setup your own complex projects pipeline and be happy.

Thanks.