Building Reproducible, Verifiable Binaries with Golang

If you just want to jump to the Github repo with 30 different build configurations, click here .

Imagine a scenario where you need to distribute your application far and wide, across the ether. People all around the world run to their computers because they heard the news that your application is finally live. They shed a single tear as they hit download, as they have been waiting for this moment for a lifetime. But wait, how will your users know that the application they are downloading was compiled from your source code, especially if your software handles sensitive data or needs a high level of security? A checksum, of course! But hang on, you might need to compile the binary on different machines, or even have the user compile the binary themselves to independently verify the integrity of the code. This is where things become a little more complicated.

There is no built-in way to compile a reproducible binary with Golang. There are specific flags that you can use during compilation like -trimpath which removes prefixes from recorded source file paths or -ldflags=-buildid= , which leaves the build ID blank, but to create a reproducible binary across every computer, you will need an identical environment. That means the same operating system, the same Go version, and the same environment variables. I wrote this article because I needed a way to create a reproducible binary so that users could compile an application from the source code and independently verify the application's integrity. After searching the internet for a way to do this, I only found forum posts that led to a dead-end or approaches that seemed to work but were much more complicated than they needed to be. So after I finally found a way to do this, I decided to share the approach so that it might save someone some time and energy. With the context out of the way, here is a straightforward way to create reproducible binaries with Go, that even the average end-user can follow.

The Walkthrough

Since we will need an identical environment to compile a reproducible binary, we are going to use Docker. If you are unfamiliar, Docker is a containerization software that makes it easy to replicate environments on any machine. We will also be creating a Makefile to abstract any Docker commands, so that all you need to do, is input one command into the terminal to compile the binary. If you are familiar with Docker and Makefiles, this entire setup will take 5 minutes.

Install Docker

First, make sure you have docker installed. If you are on a Linux distro based on Debian, you can type sudo apt-get install docker.io into the terminal and you will be good to go. Otherwise, here are the official docs to install Docker.

Creating the Dockerfile

Next, create a new file called Dockerfile and paste in the code below. I’ll explain what each step is doing.

#Get golang 1.17-buster as a base image
FROM golang:1.17-buster as builder

#Set build arguments
ARG BUILDOS
ARG BUILDARCH
ARG BUILDNAME

#Define the working directory in the container
WORKDIR /app

#Copy all files from root into the container
COPY . ./

#Use go mod tidy to handle dependencies
RUN go mod tidy


#Compile the binary
RUN env GOOS=$BUILDOS GOARCH=$BUILDARCH go build -o $BUILDNAME -trimpath -ldflags=-buildid=

First, the script gets Golang 1.17-buster as a base image and then sets build arguments for the operating system, the system architecture and the filename that you want the compiled binary to be called. Then, the Dockerfile defines the working directory for the container and copies all of the files from the root folder of your project into the container. The container then runs go mod tidy to make sure all dependencies required in the source code match the go.mod file. Finally, the container runs the command to build the binary by passing in the BUILDOS, BUILDARCH and BUILDNAME arguments. The flags -trimpath and -ldflags=-buildid= are passed in to strip any file paths and remove the build ID for good measure. You might be wondering where BUILDOS, BUILDARCH and BUILDNAME come from, which is a good question and a great transition into the next step!

Creating the Makefile

For the uninitiated, a Makefile contains shell commands that execute a series of predefined instructions. In our case, we will be building a makefile to compile binaries for each operating system. First, create a new file called Makefile and paste in the code below. Again, I will explain what each step is doing.

BUILDNAME=yourBinaryName

linux-binary: 

	#Build the linux-binary image
	docker build -t linux-binary-image --build-arg BUILDOS=linux --build-arg BUILDARCH=amd64 --build-arg BUILDNAME=$(BUILDNAME) --no-cache .

	#Run the linux-binary-image in a new container called linux-binary
	docker run --name linux-binary  linux-binary-image

	#Copy the compiled linux binary to the host machine
	docker cp linux-binary:/app/$(BUILDNAME) .
	
	#Remove the linux-binary container
	docker rm linux-binary

	#Remove the linux-binary-image
	docker image rm linux-binary-image

First, the script creates a docker image and passes in the environment variables for the operating system, system architecture, and output filename. Next, it tells docker to run a container with that image. Then, it tells docker to copy the binary created in the container and place it in the root project folder. Afterward, it removes the container and lastly removes the image.

To execute the script on Linux or Mac, simply type sudo make linux-binary in the root folder of your project. For Windows, you will need nmake or equivalent installed and then you can run nmake linux-binary. Make sure to set the BUILDNAME to your desired filename for the binary.

And That’s It!

Now you have a way to create fully reproducible, independently verifiable binaries with Golang! If you want to integrate this functionality into your project, you can check out the source code at github.com/0xKitsune/Reproducible-Go-Binaries where there are 30 different build configurations. Thanks for reading and I hope this helped!